For years, we bought the cone of uncertainty for estimation—that is, our estimates were just as likely to be over as under.
Laurent Bossavit, in The Leprechauns of Software Engineering, shows us how that assumption is wrong. (It was an assumption that some people, including me, assumed was real.)
This is a Gaussian (normal) distribution. It’s what we expect. But, it’s almost never right. As Laurent says,
“Many projects stay in 90% done for a long time.”
What curve do our estimates follow if they don’t follow a Gaussian distribution?
Troy Magennis, in “The Economic Impact of Software Development Process Choice – Cycle Time Analysis and Monte Carlo Simulation Results,” suggests we should look at the Power-Law (Weibull) distribution.
What this distribution says with respect to estimation is this: We are good at estimating small things. We get much worse with our estimation quickly, and for the long tail (larger and larger chunks of work), we are quite bad.
Why? Because creating software is innovation. Building software is about learning. We better our learning as we proceed, assuming we finish features.
We rarely, if ever, do the same thing again. We can’t apply precise estimation approaches to something we don’t know.
You should read Troy’s paper because it’s fascinating. It’s well-written, so don’t worry about not being able to understand it. You will understand it. It’s only 10 pages long.
The question is this: What effect does understanding an estimation model have on our estimates?
If we know that the normal distribution is wrong, then we won’t apply it. Right, why would you do something you know to be wrong? You would not estimate large chunks and expect to have a +/- 10% estimate. It doesn’t make sense to do that.
But what can we do? On the printed paper, what the proceedings will show p. 5061, Troy has a table that is telling. In it, he says that if you have large unique work items or you have large WIP, you will have poor predictability. (He has suggestions for what to do for your process.)
My suggestions for your estimation:
- Estimate small chunks of work that a team can complete in a day or so.
- Keep WIP low.
- Replan as you finish work.
- Watch your cycle time.
- No multitasking.
What should you do when people ask you for estimates? What kind of requirements do you have? If you have large requirements, follow my advice and use the percentage confidence, as in We Need Planning; Do We Need Estimation? Read the estimation series or get Essays on Estimation.
You can predict a little for estimates. You can better your prediction. And, you may have to predict a large effort. In that case, it helps to know what distribution model might reflect your estimate.
