One of the things I did as part of the metric collection was look at the number of open bugs in the first release of a version (which should be the Release Candidate, but unfortunately, lately it isn’t) and the final release of a version.
What causes extra releases?
- Bugs.
- Last minute changes from the customer.
- Bugs introduced while you are adding last minute changes from the customer.
- Other stuff (mostly relating to the top 3)
So it turns out in two cases – one with 3 extra “final” releases, and the other with 5, the last release looks much worse than the first one. The one with the 5 had 50 bugs more.
Why is that?
Mostly, there’s the fact that after you (finally) released, you would stop testing. And then there wouldn’t be any more bugs. But if there are additional releases, testing continues, and more bugs are found. Since “we need to ship this release already”, only the important ones are fixed (and if bugs were the cause for an additional release, this would be those that triggered it). The rest of the bugs go into the open bug list in the release notes.
So let’s say finally the customer got what he wanted. What would they say about version X?
- I wasn’t happy with the first release
- It took 5 more releases to get where I wanted to be in the first place (and more money too…)
- And the last one was even more buggy than the first one.
I haven’t checked Wikipedia, but I think this comes close to customer dissatisfaction.