Calibration Diary

A semester-long record of ranges committed before you knew the answer, audited at the end for pattern rather than for how often you were right

Due: 28 May 2027 · Weight: 45%

This is not a record of how often you were right. It is a record of how firmly you held each range, kept in a form you can check later — because a single hit or miss tells you little, and fifteen, read together, may reveal a pattern that no one of them could.

What starts it#

Week 4 gives you the form: a low bound, a high bound, a confidence in plain words, and one line naming what would make the range miss — written before you look. Every entry in this Diary uses exactly that form, every time, recorded before the outcome is known. An entry written or adjusted after you have seen so much as a hint of the answer is not an entry; it is a guess dressed up afterwards, and it will read as one.

What counts as an entry#

Before the answer resolves, an entry records five things: the quantity, the low bound, the high bound, a stated confidence in plain language, and the miss-line. After it resolves, three more are added: what actually happened, whether your range held it, and one sentence on what the hit or miss taught you.

Where the entries come from#

Four are supplied directly by the course, and nothing else may stand in for them:

• the Week 4 rainfall commit-then-look (1 entry);
• Week 8’s two round estimates — recorded as full entries (low bound, high bound, confidence, miss-line, and the BITRE airport traffic series as resolution source) before you fold each slip and hand it in; nothing about the Week 8 session itself changes, this instruction belongs to the Diary (2 entries);
• Week 10’s single sealed forecast, committed in that session and opened only in Week 12 (1 entry).

The remaining eleven are yours: made on your own initiative, spread across Weeks 4 to 12, on any quantity you like, provided you can name the source that will resolve it before you commit to a range.

Fifteen entries minimum. At least twelve resolved by the time you submit. The four supplied entries resolve on the course’s own timetable, so meeting the twelve-resolved minimum also depends on your own self-initiated entries: at least eight of your eleven need to resolve before the due date, not just be written down by then.

What Week 12 adds#

Week 12 is the working session in which you retrieve every entry, mark each one hit or miss by plain counting — no formula, borrowing at small scale the verification idea the Bureau of Meteorology applies to its own stated confidences across very large numbers of forecasts — and look for a pattern in your misses. A peer reads your diary, searches it for what would undermine the pattern you claim, and writes one note: a pattern you missed in your own record, an entry that genuinely does not fit, or — where the claim survives a real search — the limits of what it covers and the evidence that would most threaten it. That note is part of what you submit.

What you submit#

1. The complete diary — every entry, every field, in order.
2. A plain-count calibration audit: how many of your resolved entries held the true value, against what you said your confidence was.
3. A pattern claim, about 400 words, naming what your ranges have in common as a set and citing at least four dated entries as evidence.
4. A counter-case, about 250 words, examining the entries that do not fit your claimed pattern and saying honestly whether the pattern survives them.
5. A forward commitment, about 150 words: one specific change you will make to how you write your next range, and what you would see if it worked.
6. Your Week 12 peer’s note.

What is marked#

Whether your entries show genuine precommitment — bounds and a miss-line written down before you knew, not reconstructed to look tidy afterwards. Whether your ranges are honest: wide enough to be a sincere report of what you knew, not so wide that nothing could have missed. Whether your miss-lines name a specific, checkable condition rather than a general hedge. Whether your pattern claim is actually supported by the entries you cite, and whether your counter-case engages the ones that complicate it rather than quietly leaving them out.

What is not marked#

How often you were right. A well-calibrated 75% confidence should miss about a quarter of the time, so a perfect hit record is not itself evidence of good calibration. Markers will look at whether your ranges were informative and honestly precommitted, or simply wide enough to catch almost anything; neither answer is settled by the hit count alone. There is no Brier score, no calibration curve and no statistical formula anywhere in this brief. The audit is a count you can do by hand, and everything that follows it is written reflection, not computation.