The Bayesian Paradigm, Part 1
We'd prefer to base our "new data" on something as unambiguous as possible. One way is to pose a yes-or-no question, the answer to which is the new data point. One such question: Did the project deliver its first major milestone products on the promised date?
Some organizations adopt the following strategy: If the answer is yes, continue the project. If no, cancel it. That's easy, but perhaps not a good idea: It gives all the weight to the new data and ignores the previous baseline estimate. If you initially thought you had a 90% chance of success and then missed the first milestone by a couple of days, would you automatically cancel the project?
Consider the first major milestone as a test, which we might also call an indicator or a predictor. How good is the test? The answer determines the weight to assign relative to the current estimate.
To do this, construct a 2x2 matrix using the terminology shown in Figure 2:
Figure 2: The 2x2 matrix terminology
Consider the green cells. Here the test has done its job: Projects that achieved the milestone succeeded, and those that didn't did not. We would expect most projects, but not all, to fall into these two cells. If the test was a perfect indicator or predictor, all projects would be found there.
Locate the upper-right red cell. In Bayesian lingo, this is called a false positive; most of us would call it a false alarm, like having your smoke detector go off when there is no fire. The test gets it wrong; the milestone is met, but the project ultimately fails. The milestone is a poor predictor in this case, and we expect a small number of cases in this cell.
In the lower-left, we have the opposite problem, the false negative. Here the first milestone is not met, but the project continues and is ultimately successful. Once again, the milestone has been a poor indicator. We would expect to see a small number of cases here as well.
We have all seen both false positives and false negatives. Their number weakens the reliability of the test, making the first milestone less predictive of ultimate success. However, making the milestone should always increase the estimate of the probability of a successful outcome, and missing it should always decrease it. How does the number of false positives and false negatives determine how much to raise or lower the estimate?
|
Two Approaches to a New Estimate
