Quantification of Uncertainty, Probability and Subjectivity
For myself, the following checklist of questions helps me decide whether I need to quantify subjective uncertainty:
- Is the uncertainty significant? If so, assessing a range of values will make more sense, and be easier, than specifying a single-number best guess.
- Does the uncertainty make a difference? Try varying the uncertain quantity across the range of possibilities. The uncertainty only matters if you would want to make decisions differently depending on the actual value.
- Do the experts know anything that leads them to believe some possibilities are more likely than others? I've never encountered a situation where the experts didn't have relevant beliefs, but if I did, there would be good reason to choose a uniform probability distribution where one assumes that all possibilities are equally likely.
Once the beliefs about the uncertainty have been expressed in a probability distribution, a probabilistic analysis is usually not much harder than the corresponding analysis without probabilities. It only becomes more difficult if the experts know a great deal that can be represented in models and that can be used to improve the quantification of uncertainty.
"But," people say, "the probabilities are subjective!" Actually, everything associated with decision-making is subjective, but in the interest of space I won't get into those arguments! The importance of subjective probabilities is that they provide a way to encode what may be the most important knowledge held within a company.
Consider the concept known as the "knowledge-based theory of the firm". The theory argues that knowledge is the only strategically important firm resource. Other resources, like electric power and raw materials, are available at essentially the same prices to all competitors. Knowledge is the only resource that can provide a real advantage. According to the theory, fundamentally what firms do is "apply knowledge to the production of goods and services."
Since knowledge is held by individuals and not the organization, "the central role of the enterprise and its management is to integrate distributed knowledge and make it usable".[2] Probabilities provide the best-known means for capturing precisely and in a useful fashion what the company's experts believe about the inherently uncertain business environment. Probabilities encode the information in a way that can be understood and used throughout the organization. Thus, characterizing uncertainties with probabilities would appear to be critical to an organization making best use of the knowledge held by its specialists.
Although judgmental probabilities are indeed subjective, it is important to appreciate that they are not arbitrary. If a project manager says there is a 25% chance that the project will go over budget, that manager is saying that the degree of confidence in achieving the budget is the same as randomly selecting a red ball from an urn containing one red ball and three white balls. Thus, subjective probability is related to an objective reality. Expressing uncertainty as a probability gives a much more precise and useful statement than saying "it's uncertain." Furthermore, judgmental probabilities can be calibrated to experience. If there is ample evidence that only one-fourth of projects come in on budget, then, presumably, others will have more confidence in the 25% probability judgment.
The existence of uncertainty does not undermine the usefulness of probabilistic methods. On the contrary, it enhances their usefulness. When significant uncertainties are present, only a systematic and rigorous approach can produce an accurate understanding of risk and support a sound logic for making risky decisions.
Beating the 60% Solution
The 60% solution can be beaten! It may not be easy, but it can definitely be done. The fact that optimizing project decisions is hard to do, but doable, is why organizations that address the problems identified in this paper are successful. In the way described, they create for themselves a significant competitive business advantage.
2. R. M. Grant, "Toward a Knowledge-Based Theory of the Firm," in the Strategic Management Journal, Vol. 17, 1996.
|
Why Isn't Everyone Doing This Already?