Published here July 2012

Musings Index

Measuring and Estimating Progress

According to Joseph Heagney, in his book Fundamentals of Project Management:[1]

"One of the hardest things to do in managing projects is to actually measure progress. When you are following a road map, you monitor the road signs and see whether they agree with your planned route. In well-defined jobs, such as construction projects, it is generally fairly easy to tell where you are. You can measure the height of a brick wall or see whether the entire conduit is installed, and so on. That is, you can tell where you are when a part of the work is actually finished.

[But] When work is poorly defined and it is only partially complete, however, you have to estimate where you are."

Joseph goes on to explain that this is especially true of knowledge work, that is, work done with one's head, rather than one's hands. We like to refer to this distinction as "Brain work versus Brawn work".

"If you are writing software code, designing something, or writing a book, it can be very hard to judge how far along you are and how much you have left to do. Naturally, if you can't tell where you are, you can't exercise control."

Joseph makes a very important statement here, one that is often overlooked by many people. In fact, it may be so hard that people don't even bother to try or, if they do, they do not like to commit themselves! So Joseph offers this interesting little imaginary dialogue with an anonymous individual (ANO):

Joseph goes on to argue that:

"Remember, if you have no plan, you have no control, and if you don't try to monitor and follow the plan, you definitely don't have control. And if you have no control, there is no semblance of managing. You're just flailing around!

What is important to note, however, is that some projects are capable of tighter control than others:

• Well-defined work that can be accurately measured can be controlled to tight tolerances.
• Work that is more nebulous (e.g., knowledge work) has to allow larger tolerances.

Management must recognize this and accept it.

Otherwise, you go crazy trying to achieve low percent tolerances. It's like trying to push a noodle into a straight line or nail jelly to a wall."

To emphasize:

The difficulty of measuring progress does not justify
the conclusion that it shouldn't be done.
You cannot have control unless you measure progress.

We discussed this same subject with Tom Mochal, particularly the accuracy of the baseline estimate and when that should take place in IT projects. Amongst other things, Tom suggested:

1. "Estimate the work to within 15% when you charter and schedule the project. This is the traditional approach, and in many/most cases, it is still viable. However, there is an underlying assumption that the project manager and/or the project team have done this kind of work before.
   
   They should therefore be able to estimate the work to within 15% based on the high-level requirements that were gathered as a part of creating the project charter. If you discover that you have estimated incorrectly after the requirements are gathered, you have to raise a flag at that time and ask for more money. Of course this same check needs to be done at the end of every project phase regardless of the techniques used."[2]
   
   Or:
    
2. "Estimate the work at a higher level first and then firm up after the requirements are gathered. This is a variation on the first technique above. In this approach, the project manager provides a 'best guess' estimate of the work at the same time the charter and schedule are created. This estimate may be within +/- 25%.
   
   However, based on the rules of the organization, this is not the estimate where the project manager is accountable (unlike the first option above). This estimate is just a best guess given the information at the time. After the requirements are completed, the project manager provides a more detailed estimate of the work within 15%. This is the number that the project manager is held accountable for."[3]

We had some concerns over this advice, and responded that we thought that committing to an estimate of 15% in the IT world is highly optimistic. We think that this likely accounts for the ostensibly high rates of IT project failures, when it is a lack of understanding of the estimating process that is the problem. We pointed out that our "Estimating Accuracy Trumpet"[4] (for construction projects) is even higher than this at the "Charter" stage, i.e. end of "feasibility", which is usually only about 15% of the eventual total known information.

To which Tom replied:

"Thanks for your feedback. I should preface this by saying that our charter [i.e. the TenStep Charter] is created after the planning process, so it is a combination charter/scope statement in PMBOK terms. It is not done after initiation as in the PMBOK® Guide. So, at the point in time where you have the Charter/Scope Statement (PMBOK terms) and the schedule completed, we believe it is feasible to have the project estimated to +/- 15%.

Of course, the estimate is validated after you have gathered detailed requirements, and at all subsequent milestones. It is true that many IT projects exceed their estimates by a large percentage. However, this is a feature of not knowing how to estimate well - not because of the 15% rule.

If the team cannot provide a confident estimate within 15%, they should just create a project to gather detailed requirements. This first project can be estimated at +/- 15% and then enough information will be known to create the second project (i.e. design/construct/test) that will also be +/- 15%.

In fact management generally will not accept a budget estimate much greater than that. If you told your sponsor your estimate was +/- 30%, they would probably not consider that to be reasonable."

In other words, we are really looking at two estimates. The first is an estimate for a project to discover all the information we need to develop a second estimate for a second project to deliver the end product. This approach is quite common in the engineering-procurement-construction (EPC) industry, where a feasibility or prototype project is conducted to see if the end objective is practical and viable.

But back to the IT "knowledge" industry, if all of our contributors' observations are true, then it suggests to us that it is high time that we started getting senior managers of IT better educated. That is, if they want better control and fewer projects labeled as "failures" simply because of expenditure overrunning estimates with unrealistic contingencies!

Home | Issacons | PM Glossary | Papers & Books | Max's Musings
Guest Articles | Contact Info | Search My Site | Site Map | Top of Page