Week 6: Ship It, Then Pay the Hundred-x

Manufacturing's own quality-cost data spells out the thesis as an actual multiplier — catch a defect in design and it costs one dollar, catch it after the customer does and it costs a hundred

29 March 2027

Open the slides

Every other week in this course has had to argue its way to “the last mile is disproportionately expensive.” Manufacturing reaches for the accounting instead — quality engineers classify cost into prevention, appraisal, and failure, and the specific multiplier chart everyone reaches for (1x at design, 6.5x at implementation, 15x at testing, 60 to 100x after release) gets credited to something called the “IBM Systems Sciences Institute.” Here’s the complication worth admitting before leaning on it: that institute’s own underlying study has never been produced. Software researchers Hillel Wayne and Laurent Bossavit traced the citation trail back through decades of textbooks and found it dead-ends in 1981 internal IBM training notes with no public dataset behind them — a number that got laundered into “settled research” by being repeated, not by being re-measured. A course built on finishing things properly shouldn’t cite a stat that never finished being checked.

What survives that scrutiny is smaller but real: Boehm and Basili’s peer-reviewed 2001 IEEE Computer paper found that fixing a defect after delivery runs roughly 100 times more expensive than catching it during requirements and design on large projects — while adding their own caveat that the ratio drops to closer to 5:1 on small, non-critical ones. The order of magnitude survived independent replication. The specific chart with the impressive-sounding institute’s name attached to it did not.

The reason this multiplier exists is the same reason software’s last mile compounds, but manufacturing makes it literal instead of architectural: every stage a defect survives, it gets buried under more finished work built on top of it. A flawed part caught on the design bench is a sketch you redraw. The identical flaw caught in a shipped product is a recall, a warranty claim, a support call, a reputational hit, and — if it’s mechanical rather than digital — physical units already sitting in warehouses and driveways that all need to be found and touched again.

That slider is deliberately unit-less, and it’s worth saying why rather than quietly picking a currency. Every dollar-denominated version of this multiplier we could find traces back to the same IBM training-note lineage already flagged above as unverifiable — putting a dollar sign on it would just launder the number a second time. Boehm and Basili’s own replicated figure isn’t costed either: they measured a ratio of relative effort, not a price. NIST’s 2002 report does contain a real, defensible dollar figure — software defects were estimated to cost the US economy $59.5 billion a year, with $22.2 billion of that recoverable through better testing infrastructure — but that’s a national, economy-wide aggregate, not a per-defect or per-phase cost, so it can’t be repurposed to build a curve like this one without inventing the missing step ourselves. So the honest choice is the unit-less one: the slider reads in relative cost, stated plainly as an index rather than dressed up as a figure it isn’t.

Here’s the complication worth sitting with: even the honest, surviving version of this multiplier is exactly why “finish inspection later” is never a rational cost-saving move in manufacturing, even though it looks like one on this quarter’s budget. Skipping appraisal costs early doesn’t remove the cost of the defects that were there anyway — it just relocates that cost to the far end of the curve, where the peer-reviewed order of magnitude is still roughly a hundred, and now includes customers. The thesis usually shows up as “finishing was more expensive than expected.” Manufacturing’s version is sharper: finishing wasn’t more expensive than expected, it was exactly as expensive as Boehm and Basili’s replicated finding said it would be — the surprise was only in not multiplying it out in advance, and in how long a made-up institute’s name got away with standing in for that real result.