Week 5: Build the first instrument

Construct the course's seven-rung ladder from performances you have already read, find out where it cannot decide a case, and locate the assumption it quietly makes.

22 March 2027

Open the slides

Two choices in building this ladder#

Week 2 used recognition to mean a feeling of familiarity. Rung 1 now requires an observable recognition response; that is limited evidence, not the feeling alone. Application and connection were candidate performances in our plan. Here we judge them through the defined rungs—for example, explaining a new case or justifying a connection—rather than making them separate rungs.

Retrieve your Week 4 rule · 2 minutes#

Bring back the sentence “I counted that as stronger evidence because…” exactly as you wrote it. After learning the seven rungs, identify the rung your rule rewards and one performance it cannot distinguish. Preserve the original, then write a revised rule. If you have no record, use this provisional rule: “I counted a correct answer without the reference as stronger evidence.”

Can we systematically distinguish stronger and weaker evidence of understanding?

This is a self-guided lesson. These notes and the slide deck each contain the explanations you need; no presenter is assumed. Allow about 40 minutes, with another 25 if you complete the Week 5 investigation. Use paper or a text document. You need nothing from Weeks 1 to 4 open in front of you — every performance this lesson uses is quoted again below — but it helps to have read them once. Sam, Theo, Priya and Rowan are the fictional people from those weeks; nothing new is invented about any of them here.

What you will learn#

By the end you should be able to state the seven rungs of the course’s ladder with an operational definition and a real example for each; apply it to a performance you did not produce yourself and expect to disagree with another reader about a genuine case; name one rung the last four weeks never produced evidence for, and say why that gap matters; and state the assumption the ladder makes by how its own rungs are worded, which next week attacks directly.

1. Where Week 4 left us#

Week 4 gave one person, Rowan, three performances on one lock under three conditions, and found that two careful readers could reach opposite verdicts from the same texts with no way to show each other where they went wrong. It closed by naming what was missing: not more evidence, but a rule for weighing the evidence already in hand — “a rule, written down before you look, saying what counts as stronger and weaker evidence, so that it can be inspected, applied by someone else, and argued with.” Week 4 also previewed this loosely, as a ladder running “from recognising something up to connecting it to other things.” That phrase was a placeholder; naming the actual top rung is this week’s job, not last week’s. Here is the precise version.

2. What an instrument is, and is not#

An instrument, as this course uses the word, is a stated rule for turning a performance into a judgement, written before you look at the case it will be applied to. It is not a theory of what understanding is — it does not explain why some performances are harder to produce than others, and it makes no claim about what happens inside a learner’s head. It is a tool for making your own judgement inspectable: so that when you and another reader disagree, you can each point at which rung you assigned and argue about that, rather than trading impressions of how convincing someone sounded.

Week 1’s evidence record (“Observation / Interpretation / Limit”) was already a small instrument, and Week 3’s four-letter annotation scheme (P / R / C / X) was a second one, built for one specific kind of performance. This week’s ladder is broader: a single scale you can apply to a recognition, an explanation, a prediction, almost any performance the course produces from here on.

3. The ladder, rung by rung#

Each rung below is defined by what a performance has to do, not by how confident, fluent or long it is. A short, hesitant performance can reach a high rung; a long, fluent one can fail to reach a low one. Section 5 tests that claim directly.

1. Recognition — “I’ve seen this.” The person correctly matches or repeats something already stated, with it available to consult or freshly in view. This is the weakest rung: it requires identifying, not producing.

2. Recall — “I can retrieve this without the source in front of me.” The person produces a specific piece of the material — a term, a fact, a step — from memory, with nothing to consult. Recall returns material; it does not yet require the material to be correctly related to anything else.

3. Reconstruction — “I can rebuild the relationships between the parts.” Week 3’s definition, unchanged: naming the parts, placing them relative to one another, and giving the causal steps that make the thing work, without the source in view. A reconstruction needs recall to supply its pieces, but recall alone does not reach this rung — a list of correct terms with nothing joining them is Recall, not Reconstruction.

4. Explanation — “I can make this intelligible to someone else.” The person produces the structure for an audience: recast, ordered, and pitched so that another person — not just the explainer — could follow it. This is a different demand from Reconstruction, which only has to satisfy the person producing it.

5. Justification — “I can support the claims inside my explanation.” Not only asserting a causal step, but giving the reason that step holds — the “because” a listener could doubt and ask about. An explanation can be complete and still stop short of this: it can state that something happens without defending why it must.

6. Challenge — “I can respond when someone attacks the assumptions, evidence or relationships in my explanation.” Someone else raises a specific objection or counterexample, and the person answers it — by defending the original account, revising it, or showing why the objection misses. This rung cannot be produced by a person working alone; it requires another party to supply the attack.

7. Boundary — “I know where my explanation stops working.” The person names, unprompted, a condition under which their own account would fail, become uncertain, or need to be checked — the edge of their own model, found by looking rather than told by someone else.

The seven rungs are this course's own construction, built for this course's question. They are not drawn from Bloom's taxonomy or from any other published scale, and they are not claimed to be one.

4. Two things the ladder is not#

It is not strictly ordered. A performance can show Boundary-level awareness (“I’m not sure this would survive being checked”) without ever cleanly showing Justification — section 6 has an example. And it is not a checklist every task must complete: a short technical question may have no occasion for a Challenge or a Boundary at all, and that is not a failure of the person answering it. Treat the rungs as seven different kinds of evidence a performance can supply, not seven steps of a staircase everyone must climb in order.

It is also not settled. Later weeks attack the assumptions built into it — this week names one of those assumptions in section 8, and Week 12 returns to attack the ladder itself, once you have used it for a term.

5. Pause and classify · 8 minutes#

You are the examiner again. Below are five performances, all from earlier weeks, quoted exactly as they appeared. For each, write down: which rung it reaches, and one sentence defending that choice against an obvious objection. Attempt all five before reading section 6.

A. Sam, Week 1, reading the reference with it open: “Wi-Fi and internet access are different.”

B. Theo, Week 2, writing from memory with nothing in front of him: “The slider pulls the two sides of teeth together and they interlock like fingers. Pulling it back down separates them again.”

C. Priya, Week 3, reconstructing the lock from memory: “A door lock has a barrel that turns when you put the right key in. Inside there are pins of different heights. The key pushes the pins up to the right heights, and then the barrel can turn and the door opens. A wrong key does not push them to the right heights, so it will not turn.”

D. Rowan, Week 4, condition (b), explaining to a novice with the reference closed: “Every pin is a different length and every notch is a different depth. The right key lifts each pin by exactly its own amount, so all of them come flush with that join at the same moment.”

E. Rowan, Week 4, condition (c), predicting an unfamiliar case: “I think it does not matter that the other five are fine. One is enough. Because the barrel is one piece… Would you even be able to tell this key from a totally wrong one, from the outside? I do not think you would. That feels wrong to me but I cannot see what would make it false.”

6. Worked response: five performances, five disagreements#

A. Sam looks like Recognition, and a strict reading holds him there: the source was open, and repeating a stated sentence with it in view does not yet require retrieving anything unaided. But a looser reading calls it Recall, because Sam produced the sentence rather than pointing at it. The disagreement is genuine and it is exactly the kind the ladder is built to make visible: notice that it turns entirely on whether “the source was available” changes which rung applies, or only how much we trust reaching it. Hold that question; section 8 comes back to it.

B. Theo was writing with nothing in front of him, for a lesson that assumed a naive reader, so the task he attempted was Explanation. Whether his performance reaches it is a separate question, and Week 2 already answered it for us: his sentence names an outcome (“they interlock like fingers”) without the mechanism that produces it, and omits what holds one tooth to the next. A generous reading still credits Recall — he retrieved real, correct material — but the missing causal step is precisely what Explanation requires and Theo’s sentence does not supply. Fluent and confident is not the same test as complete.

C. Priya attempted Reconstruction — nothing was in front of her, and the task asked her to rebuild relationships, not list terms. Her performance, though, is thin at exactly the place Reconstruction is defined: four parts are missing, and the one relation she does state (“right heights, so it turns”) is a rule with no mechanism behind it, which Week 3 called a missing causal step. A reader who scores by task attempted places her at Reconstruction; a reader who scores by what actually got reconstructed drops her toward Recall, since what survives is closer to a remembered outcome than a rebuilt structure. Both readings are defensible, and neither is obviously right.

D. Rowan’s (b) is Explanation by task — a source-free account, pitched for someone with no vocabulary for locks — and it is fluent, complete in outline, and wrong in one place: the two-pin stack has collapsed into a single pin, which Week 4 showed is not a slip of phrasing but a change to the mechanism itself, since it removes a whole failure mode. So this performance shows more than Theo’s — it has structure, an audience, and independent phrasing — and it still does not establish the mechanism correctly. Reaching a rung is not the same as reaching it accurately; the ladder tells you what kind of evidence you have, not whether that evidence is good news.

E. Rowan’s (c) is the hardest of the five, on purpose. The middle sentence — “one is enough… because the barrel is one piece” — gives a reason for a claim, which is Justification: it says why, not just that. The closing sentence — “would you even be able to tell this key from a totally wrong one… I cannot see what would make it false” — is Boundary: Rowan is naming, unprompted, the edge of what this performance can show from the outside. Notice what is missing in between: nothing here is Challenge, because nobody attacked Rowan’s account and made them defend it — the hedged opening (“do you want me to guess?”) is not a challenge, it is uncertainty about the genre of the task. So this one performance touches rungs 5 and 7 without ever passing through 6, which is exactly the “not strictly ordered” claim in section 4, demonstrated rather than asserted.

7. The rung nobody produced#

Read back across all five. None of Sam, Theo, Priya or Rowan across all three conditions ever answers an objection someone else raised. That is not a coincidence of a small sample — across four weeks of this course, nobody has yet been shown a counterargument to their own explanation and asked to respond. Challenge cannot be produced by a person working alone, however hard they think about their own account; it needs a second party who has not already agreed with them. That is a real limit on what the last four weeks of evidence can tell you, and it is also a preview: an instrument that only measures what a solo performance can supply is missing a whole rung, and the course will need an activity that specifically manufactures one.

8. Pause and write: produce a Challenge · 6 minutes#

Take the explanation you wrote in the Week 2 or Week 3 investigation — your own zip or lock account, or the second mechanism you tried in the solo route. If you did not keep one, use Priya’s reconstruction in section 5.

Here is an attack on it: “Your account never says what happens if two parts are both slightly wrong at once — does the mechanism fail twice as badly, or is one wrong part already enough that a second one changes nothing?”

Write your response in three or four sentences. You may revise your original claim, defend it as it stands, or say precisely what you would need to check to answer the attack. Attempt this before reading the worked response.

9. The assumption the ladder hides#

Look again at rungs 2 and 3. Recall is defined as producing material “without the source in front of me.” Reconstruction is defined as rebuilding relationships “without the source in view.” The absence of a resource is not incidental to these two rungs — it is written into their definition. A performance produced with a reference open cannot, by this wording, count as Recall or Reconstruction at all, however accurate it is.

That is a real assumption, not a neutral description, and this week is not the place to defend or attack it — only to name it plainly, the way Week 4 asked you to name the rule behind your own verdicts. The assumption is: a performance produced without support is intrinsically stronger evidence than the same performance produced with support. Section 6’s disagreement about Sam already turned on exactly this: does the source’s presence change what rung applies, or only how confidently we can trust it? Next week attacks that assumption directly, by holding the mechanism fixed and asking the sharper question: does needing support show that you do not understand, or only that you were allowed to use what was available?

10. Keep a short record#

Before you finish, write four lines and keep them.

The second line matters most. If you agreed with every placement in section 6, go back and argue the other side of one of them before moving on — the ladder is only doing its job when it gives two careful readers something specific to disagree about.

Reading with a purpose#

Primary empirical source, reused. Rozenblit, L., and Keil, F. (2002). “The misunderstood limits of folk science: an illusion of explanatory depth.” Cognitive Science, 26(5), 521–562. Full text from the authors’ lab. Weeks 2 and 3 used this for the gap between a fluent explanation and a complete one. This week’s rungs 4 and 5 — Explanation and Justification — are one way of making that same gap into two separately checkable claims rather than one vague one. This is an empirical finding, not a theory, and Study 1 alone rests on sixteen graduate students at one university.

Theoretical argument, reused. Clark, A., and Chalmers, D. (1998). “The Extended Mind.” Analysis, 58(1), 7–19. Read the authors’ text, sections 1 to 3. Section 9 above names an assumption the ladder makes about resources without yet arguing it either way. Before Week 6, read (or reread) Clark and Chalmers’ case that an external resource can participate in thinking rather than merely assist it, and ask which of the seven rungs, if any, that argument would change.

Both readings are carried over because they do new work for this week’s question rather than being repeated out of habit; neither is required to complete the lesson.

What this leaves unresolved#

The ladder makes disagreement inspectable — you can now say exactly which rung you placed a performance at, and someone else can argue you out of it, which was not possible in Week 4. It has not made judgement automatic, and section 9 found something sharper than an open question: a specific assumption sitting inside the wording of two of its own rungs, unexamined so far. Apply the ladder yourself in the Week 5 investigation, which asks you to classify performances of your own and produce the Challenge-rung response this lesson could only demonstrate once. Next week uses a new courier scenario to compare what a person is allowed to consult while producing an answer — testing directly whether needing support is evidence against understanding, or evidence of nothing at all.