Week 11 Session: Why it persists

Nine years of results tables carry an FID column, and that continuity is the strongest argument anyone has for keeping it. What the number costs to leave out, stated honestly.

SLOP8412Week 1119 October 2026Arc C

Open the results table of a generative modelling paper published this year, then open one from 2019. Both carry an FID column. That continuity is the strongest argument anyone has for the number, and it is a good argument.

The case for the row

A score is only useful against other scores. Nine years of published results carry FID, computed against reference sets that are named and architectures that can be looked up. A new model reporting one can be placed in that record immediately. A new model reporting none cannot be placed at all, and a reader has no way to judge whether it beats the thing it says it beats.

Reviewers ask for it, and asking costs them nothing. The cost of answering falls on the author, and it is small: one more evaluation against a reference set they already have. The cost of refusing is a review saying the comparison is incomplete, which is hard to argue with, because it is accurate.

Recomputing baselines under a different metric is the work that would end this, and nobody is funded to do it. Every prior number would have to be regenerated from checkpoints that are sometimes published and sometimes not, by a person with no result of their own at the end of it. That work is a public good, and public goods are underproduced.

What the incentive produces

Kynkäänniemi and colleagues report that the score responds to matching the ImageNet class histogram of the samples, with no corresponding change in what a person would call quality. A model can be moved up the table by an intervention that is not an improvement.

That result is published, it is cited, and FID remains standard. The reading to take from it is not that the field is careless. It is that a metric’s known defects do not cancel the coordination value of everyone reporting the same number, and coordination is what the row supplies.

The bench, unchanged

Score A and B once more. The true values are still 5.12 and 4.88, fixed since week 2, and B is still the closer candidate. Your estimates are not those numbers, and at the sample sizes week 5 examined they need not even keep that order. Ten weeks changed what you know about the estimate. They changed nothing about the reason a researcher would report it.

The week 11 lecture makes you state the protocol behind a claim like that one, runs it, and writes the sentence the evidence will carry.

Exercise

Write the paragraph a paper would need in order to justify omitting FID: what it reports instead, why that suffices, and what a reader gives up.

Then write two sentences estimating how a reviewer reads that paragraph.

In your measurement log

This week’s entry: the paragraph justifying omission, and how you expect a reviewer to read it. It joins themeasurement log, which is read as a whole at the end of week 12.

Reading

Required

Further

The spec

What you bring to the session. Written so a reader can tell whether it has been met without asking you.

S11 / Optional self-check · Not assessed

Before you leave

No marks or penalties. Try it, retry it, or read the explanation. Nothing here is submitted.

Which is the strongest defensible reason to keep reporting FID after documenting its limitations?

Answer and explanation

Correct: C. A shared metric has coordination value: readers can place new results against established baselines. That value depends on comparability; the presence of an FID column alone does not establish it. Retaining FID does not require pretending its known blind spots disappeared.

Feedback for A: Adoption is a reason a number is useful socially, not proof of its validity for every scientific question.

Feedback for B: Standardisation of the metric's name does not make all evaluation protocols identical.

Teaching takeaway: A useful common reference can still need qualifications.