Week 6: Disagreement is information

Why splitting the difference destroys evidence, the small set of axes real disagreements sit on, and why nobody warns you at the moment you need it

29 March 2027

Open the slides

Somebody hands you two figures for the same thing. One says four hundred, the other says six hundred. What do you do in the next thirty seconds?

Most people, in most rooms, reach for five hundred.

What the average destroys

Averaging two numbers is a reasonable response to one situation: two attempts to measure the same quantity, each carrying error that could have fallen either way. Then the middle is a sensible guess and the spread tells you something about your precision.

That situation is less common than the reflex suggests. Far more often the two figures are answers to two different questions, each answered correctly. The midpoint between them is then an answer to no question at all — a number with no referent, arrived at by arithmetic that was never entitled to run.

What makes this expensive is not the wrong figure. It is that the gap between the two sources was carrying information, and averaging discards it. The distance between two careful counts tells you something real about definitions, about who gets missed, about how the world was sampled. Collapse it to a midpoint and you have converted evidence into a number, which is the wrong direction of travel.

A short list, and it is genuinely short

The reassuring thing about reconciling sources is that real disagreements between careful publishers sit on a small number of axes, and you can learn the list.

Definition — what each has declared it is counting. Coverage — who could appear in the figure at all, and who is structurally invisible to it. Method — survey, census, administrative record, model, derivation from something else. Timing — both the date and, more slippery, the grain: a moment against a period. Unit — persons, cases, events, episodes, clients, records.

Run down five axes and most disagreements resolve into “these were never the same quantity”. That answer is the reconciliation. Nothing further is owed, and the pair of figures can go on standing side by side.

Timing is the one that catches trained people, because the date is easy to check and the grain is easy to miss. A count of who is in a condition on one night and a count of everyone who passed through that condition over a year are different kinds of number. They can be close, far apart, or in either order, and none of those outcomes tells you anything until you have noticed which is which.

Nobody warns you at the right moment

Agencies document themselves reasonably well. The AIHW states plainly that its client numbers reflect agency engagement rather than the prevalence of homelessness in the community, which is exactly the caution a careless reader needs.

But the warning lives in the publication, and the mistake happens somewhere else — in a briefing note, a slide, a news story, a submission, where the two figures have been lifted out and set beside each other by someone who read neither methodology. There is no point in the pipeline where an alarm sounds. The comparison is only ever prevented by a person who chose to go back and check what each number was.

And when two sources happen to land close together, nothing prompts anybody to check at all. Near-agreement reads as corroboration. Occasionally it is; more often it is coincidence between quantities that were never comparable, and it is the version of this error that survives longest, because it never produces an argument.

Choosing, rather than resolving

The output of this work is usually a decision rather than a single reconciled figure: which source answers the question actually in front of you, and what that source will not support.

That second half is the professional part. A source chosen well still has a boundary, and saying where the boundary falls — this figure will carry a funding case but will not carry a claim about a trend, this one describes service demand but not need — is the difference between using evidence and citing it.

Outline