Defining Project Success within a Range
Introduction
Tom Mochal of TenStep, Inc. a long-time friend of mine, writes a regular column in which he dispenses his wisdom in Tip of the Week.[1] In particular, on 12-22-2010 he wrote on the subject of project success and this got us thinking. Here is what Tom said, followed by our own opinion.
Tom said:
Was Your Project Really Successful?
You have all read the stories about the large number of projects that fail. Depending on the report you read, half or more of projects fail - perhaps as high as 80%. According to the reports, the larger the project, the greater the chance that it will be a failure.
Let's assume that the number of failed projects is 50% - the low number. What is interesting is to try to relate that number to my personal experience. Where are all of these failed projects coming from? Can you say that half or more of the projects that you managed were failures?
MW: Of course this does not refer to all projects in general. It refers to a popular consulting report focusing only on information-technology (IT) projects. In our own experience of non-IT projects the success rate based on success of the outcome has been absolute. In fact we specialized in fixing failing projects on the theory that it is difficult to do worse than the previous guy and so it is easier to come out "smelling like roses"!
Tom said:
It's All in the Definition
The idea of a failed project starts with understanding the definition. You may have a perception of what it means to manage a failed project. Your company should have a definition as well, and if they do, your definition should be the same as theirs. One major concept that plays a key role is the idea of tolerances.
MW: We'll go along with that!
Tom said:
Define a Reasonable Cost and Duration Tolerance Level
If you estimate a project to cost $230,000, is your project a failure if the actual cost is $230,500? You missed your budget, right? Yes, but this gets into the concept of tolerances. If you delivered within $500 on a $230,000 budget, you should be lifted on your manager's shoulder and paraded around the company as a hero.
Your company needs to establish the tolerance level that they consider to be reasonable for projects. For example, the tolerance level may be -10% to +15%. That is, if you deliver the project 15% over budget, it is still considered a success. For the $230,000 project, that means you could have gone over budget by $34,500 and still have been considered successful. The baseline budget should also include any formally approved scope changes. For example, if your original budget was $200,000, and the client approved an additional $30,000 in scope changes, then the $230,000 is the number that you get held accountable for, plus your tolerances.
Normally there is some room for tolerances with your deadline as well. If the projects are internally focused, the end dates are in many cases arbitrary. Your original deadline must also be extended if scope changes were approved. Of course, not all projects have that flexibility. The YR2K software projects for instance, typically had to be completed by the first time they ran in the new-year. A week late was not good enough.
MW: From a scheduling perspective, someone has to set the target dates. In fact most schedule dates are arbitrary. "We think there is this much work in the activity and if we apply these resources we calculate we shall be finished by ... etc." Any of these suppositions could be wrong. More significantly, most projects are only one part of a much larger picture. Therefore there are other considerations on which the end date depends, such as follow-on projects, availability of specialized people, committed dates for business process changes and so on.
MW: As to budgeting, there is a serious problem with this philosophy. If the accepted norm is "X+Y%", very soon the real budget is seen as "X+Y%" because we know that's OK. So what if, as a matter of policy for example, we go over by say "(X+Y%) +Y% of (X+Y)" (and so on)? No, it is the project sponsor's job to see that a realistic budget is set for the work contemplated and to do this in consultation with, and the concurrence of, the project manager. And having done that, then stick to it!
MW: In our experience in the private sector, if our company was smart we might win a competitively bid contract where the resulting profit margin was typically 5%. If, as project manager, your total costs, including overheads I might add, exceeded that 5%, then your company was losing money and the project was not a success! It's amazing how that drove the desire to be effective in planning and efficient in production. It was also most irritating when people not directly associated with the project, but nevertheless seeking some favors, assumed that there were stashes of money floating around to please them!
Tom said:
Declaring Success from a Project Perspective
Once you understand your tolerances (if any), you can start to evaluate success from a project perspective. Generally, the project team members can declare success if:
- The project is delivered within the estimated cost, plus or minus the tolerance.
- The project was delivered within its deadline, plus or minus the tolerance.
- All of the major deliverables were completed. (Some minor ones, or minor functionality, might not be delivered.)
- The overall quality is acceptable. (It does not have to be perfect.)
Some companies also look at whether the project team was easy to do business with. That is, did the client and the project team work well together? For instance, was there good communication? If the client had another project (and a choice), would they ask you to work on it again? We call this "performance excellence".
Other factors may be important for specific projects. For instance in a construction project, safety might be a key success component.
A Project Scorecard can be used to establish the formal criteria for success.
MW: Tom is essentially writing from the perspective of the IT sector. But, subject to our reservations in the previous section, Tom's listing of "success" considerations we believe are just as valid for any other area. However, in the construction sector in our western world, the work forces are expected to work safely, and a high level of coordination (read "communication") is both essential and expected if the project is to be accomplished at all.
MW: We do, however, have some reservations about "the client and the project team working well together". On the face of it, this is good, but only to the extent of getting the work done. Unfortunately, as we have learned to our cost, a too cozy a relationship up and down the client/project team hierarchy can be the source of all too many "scope-change" items creeping into the work. Any hint of scope change should be strictly "off-limits" so far as the team is concerned and be properly referred to the established scope change process - however small the change or changes may be. If the project manager sees fit to facilitate multiple minor scope change arrangements into one comprehensive change order, then that is another matter – and quite acceptable!
Tom said:
Declaring Success from a Company Perspective
Declaring success from a project perspective is normally what the project team is asked to be accountable for. However, from a company perspective, success would also be based on whether the company received the business value that was promised. If the project was a failure from a project perspective, it is normally a failure from a company perspective as well. (Although this is not always the case.)
However, there are also many examples of projects that were successfully delivered, yet are not delivering the value promised. If the project team delivered successfully within tolerances, there is usually nothing else that can be done from their perspective. However, it is possible that the return on investment calculations were faulty, or the client and sponsor misjudged the marketplace. It is also possible that this project was part of a larger initiative. Although your project may be successful, the overall, larger initiative may be a failure.
We believe that success against the project business value, as defined in the Business Case, is ultimately the responsibility of the sponsor - not the project team.
MW: And with that, we entirely agree!
Tom said:
In Summary
Every organization should have some general rules about how to declare project success or failure. Your project isn't a failure if you miss the budget by a dollar and deliver a day late. Normally a project will still be considered successful if it delivers within cost and deadline tolerances, and delivers all major deliverables within acceptable quality. This is what the project manager and project team can be held accountable for.
MW: Amen to that!
1. You have to subscribe to Tom's Newsletter to receive his Tip of the Week. To do so, simply contact Tom at Tom.Mochal@TenStep.com.
|
