Cost of quality is a ledger, not a vibe#
Quality engineers already split cost into prevention, appraisal, and failure — which means manufacturing can put a real number on “the last mile is expensive” that every other week in this course had to argue its way toward.
The chart everyone in software has seen#
Cost to fix a defect, by the stage you catch it at:
| Stage | Multiplier |
|---|---|
| Design | 1x |
| Implementation | 6.5x |
| Testing | 15x |
| After release | 60–100x |
Usually credited to something called the “IBM Systems Sciences Institute.”
The problem#
“We’ll hone our scholarship skills by looking up the original source for these ideas.”
Laurent Bossavit, The Leprechauns of Software Engineering
Bossavit devotes a full chapter — “The cost of defects: an illustrated history” — to tracing this exact chart’s citation trail back through decades of textbooks. It dead-ends in 1981 internal IBM training notes, with no public dataset behind them.
What actually survives#
Boehm and Basili, 2001, peer-reviewed, IEEE Computer:
Fixing a defect after delivery runs roughly 100x more expensive than catching it at requirements/design — on large projects. Their own paper adds the caveat: closer to 5:1 on small, non-critical ones.
The order of magnitude held up under independent replication. The impressive-sounding institute’s name did not.
Cost of quality, zoomed out#
A single defect’s multiplier is one line item inside a much bigger number:
- Total quality cost — prevention + appraisal + failure combined — runs 10 to 20% of revenue in a typical, well-run firm.
- In a poorly-run one: up to 40%.
That’s the American Society for Quality’s own benchmark for the ledger this whole chart lives inside.
Still true, or just old?#
Mike Cohn argues Boehm’s curve has been flattening for decades — modern IDEs, automated testing and CI, and modular architecture each cut into it before agile even started:
“The problem isn’t that Boehm was wrong. The problem is that we treated his findings as if they were timeless.”
This is a live disagreement, not a resolved one.
The peer-reviewed number, disaggregated#
Boehm and Basili didn’t publish one multiplier — they published two, and the gap between them is the argument:
| Project size | Post-delivery vs. design-stage fix cost |
|---|---|
| Large | ~100x |
| Small, non-critical | ~5:1 |
Same defect, same category of mistake. The multiplier isn’t a property of the bug — it’s a property of how much finished work already sits on top of it by the time someone else finds it.
Where NIST’s own number actually falls#
NIST’s 2002 report splits its $59.5B/$22.2B national estimate by who absorbs it:
| Who bears it | Annual cost | Recoverable |
|---|---|---|
| Software developers | $21.2B | $10.6B |
| Software users | $38.3B | $11.7B |
Nearly two-thirds of the cost, and most of what’s actually recoverable, sits on the far side of the sale — with the people who didn’t write the defect.
Two manufacturing sectors, in NIST’s own figures#
The same report breaks its estimate down by industry. Two are manufacturing’s own:
| Sector | Annual cost | Recoverable (50% error reduction) |
|---|---|---|
| Automotive | $1,229.7M | $377.0M |
| Aerospace | $237.4M | $54.5M |
This is the closest thing in the report to a manufacturing-specific dollar figure — and it’s still an aggregate across the whole sector, not a per-defect price. It confirms the shape of the claim without letting us launder it into a false precision.
Case study: Therac-25#
Between 1985 and 1987, a radiation therapy machine delivered massive overdoses to six patients. Several died.
“A detailed investigation of the factors involved in the software-related overdoses… presents lessons learned in terms of system engineering, software engineering, and government regulation of safety-critical systems.”
Leveson & Turner, IEEE Computer, 1993
The race condition behind it existed in code carried over from an earlier, less automated model — nobody re-verified it against the new machine’s removed hardware interlocks until after it had already shipped.
Case study: Ariane 5 Flight 501#
37 seconds after liftoff on its maiden flight, the rocket veered off course and was destroyed. ESA’s own Inquiry Board, chaired by Jacques-Louis Lions, found:
“The failure… was due to specification and design errors in the software of the inertial reference system.”
The software itself was inherited, unchanged, from Ariane 4 — where the same code path had never been able to overflow. Nobody re-checked that assumption against Ariane 5’s faster trajectory.
“The extensive reviews and tests carried out during the Ariane 5 development programme did not include adequate analysis and testing.”
Case study: Mars Climate Orbiter#
The orbiter was lost approaching Mars in 1999. NASA’s own Mishap Investigation Board, chaired by Arthur Stephenson, found the root cause was a units mismatch: one team’s ground software produced thruster data in pound-seconds, and the navigation team’s software expected newton- seconds, as the interface spec required.
Both pieces of software had presumably been tested on their own. Nobody caught the mismatch until the whole system was already flying.
Case study: Boeing 737 MAX#
Two crashes, five months apart, 346 people. The US House Committee on Transportation and Infrastructure’s 18-month investigation put it bluntly:
“The MAX crashes were… a horrific culmination of a series of faulty technical assumptions by Boeing’s engineers, a lack of transparency on the part of Boeing’s management, and grossly insufficient oversight by the FAA.”
This is the organisational version of the same failure: MCAS, the flight control software behind both crashes, existed inside a certification process under competitive and schedule pressure to treat the last mile of review as a formality rather than a gate.
The formula, made explicit#
Quality engineers formalise this week’s opening claim as an actual equation, not just a category split:
Cost of Quality = Cost of Conformance + Cost of Nonconformance
Where conformance splits into prevention and appraisal, and nonconformance splits into internal failure and external failure — the same four buckets this week opened with.
“Failure to identify a given cost as a quality cost, or incorrectly categorizing it, may result in misleading decisions on actions to improve quality.”
Mustafa Shraim, Ohio University — American Society for Engineering Education, 2020
The counter-example: designing the multiplier out#
If catching a defect earlier is worth roughly a hundred times more than catching it late, the rational response isn’t just admitting that — it’s building a system that refuses to let a defect travel forward at all.
Toyota’s own description of jidoka:
“When an abnormality occurs… the machine or equipment can detect the abnormality and stop automatically, or the operator can stop the line by pulling the stop cord themselves… This eliminates the outflow of defective products.”
An andon cord is the cost curve, acted on: it forces every defect to be caught at 1x, by making it organisationally impossible to let one reach 2x.
What four unrelated sources agree on#
A peer-reviewed replication, a national economic-impact study, an engineering-education paper, and four separate accident investigations — different methods, different decades, different countries — all land on the same structural claim:
- The multiplier is real, even where the famous chart wasn’t.
- It falls hardest on whoever is downstream of the mistake.
- It compounds specifically because of what got built on top of the defect before anyone found it.
- The only design that reliably beats it is one that refuses to let a defect travel past the stage it was made in.
The chart needed its own last mile#
Manufacturing’s version of “the last mile is expensive” looked finished — a number, a source, decades of citations. It took someone tracing the citation trail instead of trusting it before any of it could actually be trusted.