Estimating and Forecasting Biases
People are notoriously poor at estimating and forecasting. They tend to naively extrapolate trends that they perceive in charts. They draw inferences from samples that are too small or unrepresentative. They routinely overestimate their abilities and underestimate the effort involved in completing a difficult task.
Several biases combine to cause us to make serious errors when forecasting future performance. One problem is overconfidence. We believe we are better at making forecasts or estimates than we really are. I've often repeated a well-known demonstration to illustrate what I call the "2/50 rule." People are asked to provide confidence intervals within which they are "98% sure" that various uncertain quantities lie. The quantities for the questions are selected from an Almanac, for example, "What's the elevation of the highest mountain in Texas?"
When the true value is checked, up to 50% of the time it falls outside of the specified confidence intervals. If people were not overconfident, values outside their 98% confidence ranges would occur only 2% of the time. As a related observation, there are many examples of famous people expressing utter confidence about things that are subsequently proved wrong. For example, Thomas Watson, Chairman of IBM, reportedly said, "I think there is a world market for about five computers."
Another relevant bias is anchoring. Initial impressions become reference points that anchor subsequent thoughts and judgments. For example, if a salesperson attempts to forecast next year sales by looking at current year sales, the old numbers become anchors, which the salesperson then adjusts (usually insufficiently) based on other factors. When things are changing rapidly, historical anchors lead to poor forecasts and, in turn, promote to misguided choices. Dramatic or easy to recall events often become strong anchors. For example, the vividness of the horrible events of September 11 caused many to view airline travel as too risky, but many experts believe that travel has never been safer.
Various motivational biases also lead to forecasting errors. The nature of the effect can depend on the individual. For example, project managers who are anxious to be perceived as successful may pad cost and schedule estimates to reduce the likelihood that they fail to achieve expectations. On the other hand, project managers who want to be regarded (consciously or unconsciously) as high-performers may underestimate the required work and set unrealistic goals.
Forecasting errors are often attributed to the fact that most people don't get good feedback about the accuracy of their forecasts. We are all fairly good at estimating physical characteristics like volume, distance, and weight because we make such estimates frequently and get feedback about our accuracy. We are less experienced (and get less verification) when making more uncertain forecasts. Weather forecasters and bookmakers have opportunities and incentives to maintain records of their judgments and see when they have been inaccurate. Studies show that they do well in estimating the accuracy of their predictions.
Another problem is that people often fail to properly consider statistical information. When forecasting how long it will take to complete a project, for example, project managers may fail to consider the time taken to do previous projects. Rather, they take an "insider's view" of the current project, thinking only about the steps and scenarios leading to successful completion. This usually results in overly optimistic forecasts. Also, people tend to be insufficiently conservative (or "regressive") when making predictions based on events that are partially random. For example, shareholders expect a company that has just experienced record profits to earn as much or more the next year, even if there have been no changes in products or other elements of the business that would explain the recent, better-than-anticipated performance.
Advice for improving forecasts and estimates includes:
- Think about the problem on your own before consulting others and getting anchored to their biases.
- Be open-minded and receptive. Seek opinions from multiple and diverse sources. Tell them as little as possible about your own ideas beforehand.
- Tell people you want "realistic" estimates. Ask about implicit assumptions.
- Encourage the estimation of a range of possibilities instead of just a point estimate. Ask for low and high values first (rather than for a middle or best-guess value) so as to create extreme-valued anchors that counteract the tendency toward overconfidence around a middle value.
- Require project proponents to identify reasons why what they propose might fail.
- Give people who provide you with estimates knowledge of results as quickly as possible.
- Use network diagrams and similar devices to identify and define the sequencing of component activities. A major value of such techniques is that they reduce the likelihood that necessary activities, such as procurement and training, aren't overlooked.
- Routinely use logic to check estimates. As a simple example, if you have 2 months to complete a project estimated to require 2000 hours, verify that you have a sufficient number of FTE's available.
|
Framing Bias
