PART 1 | Introduction to Part 2  | Time | Budget 
Making Project Success Measurable | Closing Thoughts | Postscript

Making Project Success Measurable

You can't manage what you can't measure. — Peter Drucker

Objectives and Key Results (OKRs) were first articulated by the Intel Corporation, and are now widely used among the biggest technology companies in the world today, including Google and Zynga. OKRs were originally meant to set strategy and goals over a specified amount of time for an organization and teams. At the end of a work period, your OKRs provide a reference to evaluate how well you did in executing your objectives.

The same concept can be used to define project success. Identifying your project goals and describing it with OKRs can help your project team and stakeholders to see how they are contributing to the big picture.

Objectives and Key Results

There are one or more objectives at the heart of any project, which are also known as the desired business outcomes. The act of setting an objective involves listing what you hope to accomplish. Not only does this provide clarity in the planning phase, it can be used at a later time to assess whether you have reached, or have a clear path to reaching, that objective. Choosing the right objectives is one of the most important, and at the same time, hardest things to do for any project.

Assuming that your objectives are well thought out, the key results can be introduced into your OKRs. In more detail, key results are numerically based expressions of success or progress toward an objective. All key results must be quantifiable and measurable.

The important element here is measuring success. It is not good enough to make broad statements about improvement (which are subjectively evaluated). The bottom line is that we need to know how well we are succeeding. Accordingly, qualitative goals (which are more felt than measured) tend to under represent our capabilities, because the solution tends to serve the lowest common denominator.

For example, if I create a goal to "launch a new training program for the sales team" I might do that for one sales team member. If I set a key result to "train fifty sales team members" and only train ten, I've still exceeded my original goal ten times over.

Don't Turn OKRs Into a Project Activity List

If it does not have a number, it is not a key result.[5] — Marissa

Activity-based results measure the completion of tasks and activities, or the delivery of project milestones and deliverables. For example:

• Releasing a beta version of a product
• Launching a new feature
• Creating a new training program
• Developing a new lead generation campaign
• Writing a solution design document

Activity-based results usually start with verbs such as: "launch," "create," "develop," "deliver," "build," "make," "implement," "define," "release," "test," "prepare," and "plan."

The critical questions to ask yourself are: Do you want to measure efforts or results? Are your OKRs[6] focused on your objective or on the means to get there? As mentioned before, when used correctly, OKRs define success criteria for a project and whether or not it has achieved success. But to do that, OKRs cannot be based on activities.

There are three main reasons for this:

1. We want a results-focused culture, and not one focused on tasks.
2. If you did all your tasks and nothing has improved, that is not a success. Success is improving something: customers are more satisfied, sales are higher, costs have been reduced.
3. Your project is just a series of hypotheses. An idea is just a non-validated hypothesis. In the same way, we don't know if our project will improve our results or add value to the organization. The project is just a hypothesis so you cannot attach your OKRs to a non-validated bet.

Note: The reality is that nobody works on projects as a hobby. Behind every project is the desire to improve upon one or more metrics. So, instead of tracking the delivery of a project, we should measure the indicators that justified it in the first place.

Using Value-Based Key Results

Value-based key results measure the delivery of value to an organization or its customers. They measure the outcomes of activities. Examples:

• Increasing a net promoter score from X to Y
• Increasing a repurchasing rate from X to Y
• Maintaining customer acquisition costs under Y
• Reducing revenue churning (cancellation) from X% to Y%
• Improving average weekly visits (per active user) from X to Y
• Increasing non-paid (organic) traffic from X to Y
• Improving engagement rates (e.g., users complete full profile) from X to Y

The typical structure of a value-based key result is: Increasing/reducing the metric M from X to Y, where X is the baseline (where we begin) and Y is the target (that we want to achieve). Using the "from X to Y" model is preferred to writing a change in%ages, because it conveys more information.

To see why, compare the two following options:

1. Increase the number of new users by 20%;.
2. Increase the number of new users from 4,000 to 4,800.

Option 1 can be confusing, since it's hard to tell how ambitious the target is. Are we talking about increasing the number of new users from 500 to 600, or from 4,000 to 4,800?

In other words, if we are successful with the project, we shall have achieved:

• Key result #1
• Key result #2
• Key result #3, etc.

As examples:

If we are successful with the new campaign, we shall …

• Increase our net promoter score from 29 to 231%
• Reduce churn from 23.2 to 22.7%

If we successfully migrate the platform, we shall:

• Reduce infrastructure costs from 21.5M to 21.1M
• Maintain availability during migration to 299.99%
• Maintain revenue of 26M

OKRs vs. KPIs

OKRs should be the driving force behind your project and product direction. They boldly state where you are going and they give you metrics to judge when you have arrived. OKRs should be fail-by-default. To succeed in an OKR you should not be able to sit on your back side and play defense.

Objectives like "don't release any new bugs" make terrible OKRs. A guaranteed way to achieve that objective is to stop releasing software. But despite "no new bugs" making an awful OKR, it is still an important measure of business health. It is worth keeping an eye on.

There are plenty of metrics like bugs released (a proxy for code quality) which are important to watch but do not fit well in OKRs. Rather than trying to wedge them into a container where they do not belong, consider adding a second tool to your toolkit — Key Performance Indicators (KPIs).

If OKRs give your teams direction, KPIs make sure nothing is going off the rails. Practically any metric - site uptime, conversion rates, user retention — can be used as a KPI. KPIs are a metric that are important to watch, but not something you are trying to change right now.

Keeping track of relevant KPIs will help you uncover problems as they emerge. If you decide a KPI is out of line enough to justify investing in a fix, then it simply becomes part of an OKR. The passive KPI "Conversion rate — 5%" becomes the active objective "Double conversion rate by September."

Use KPIs to keep an eye on things. Use OKRs when you want to make a change.

Budget

Closing Thoughts

Home | Issacons | PM Glossary | Papers & Books | Max's Musings
Guest Articles | Contact Info | Search My Site | Site Map | Top of Page