Defending the inference#

What makes the inference defensible?#

Validity concerns how well the evidence supports the particular interpretation and use we propose. A successful performance does not validate every claim about the person. This returns to Week 1’s distinction between an observation and an inference. Farhan is a new fictional hard case; compare it with your own Week 11 record, using the supplied fallback if needed.

How to use this lesson#

Read at your own pace; no presenter is needed. Allow around 40 minutes, with another 20 for the Week 12 investigation. You need nothing new: this week reuses the seven-rung ladder from Week 5.

This lesson does not add an eighth rung, and it does not resolve its own hard case cleanly. That is the point, not an unfinished ending.

Where the course has got to#

Weeks 1–4 built up reasons to distrust an unexamined answer. Week 5 turned that suspicion into an instrument: seven rungs, each defined by what a performance has to do, so two readers could disagree about a specific rung instead of trading impressions.

Weeks 6–11 kept testing the instrument against harder conditions: what a person is allowed to consult, a changed form, an unannounced follow-up, a recorded prior state, a collaborator, a real administered exam. The ladder survived all of it.

The rungs, briefly recapped#

1. Recognition — matches something already stated, in view. 2. Recall — retrieves it, nothing in view. 3. Reconstruction — rebuilds the relationships between parts. 4. Explanation — makes it intelligible to someone else. 5. Justification — supports the claims inside the explanation. 6. Challenge — answers an attack on it. 7. Boundary — names, unprompted, where the account stops working.

Full definitions and worked disagreements: Week 5, “Build the first instrument.”

Today’s subject#

Farhan is a fictional examinee, distinct from Sam, Nadia, Theo, Priya, Rowan and Elin. You are his examiner, following the administration standard this course has been building since Week 11: a real exam, on a real topic, with conditions declared in advance.

The topic: why the sky is blue during the day, and red or orange at sunset. Farhan is told the topic in advance, but not the exact questions — the same “notice” distinction Week 8 introduced.

The background Farhan is examined on#

Sunlight reaching the atmosphere is scattered in all directions by the gases in the air. Blue light, which travels as shorter, smaller waves, scatters more than the longer-waved colours — so during the day, scattered blue light reaches your eyes from all over the sky.

Near sunset, sunlight has to pass through far more atmosphere to reach you. Even more of the blue gets scattered away along that longer path, leaving mostly the reds and yellows to arrive directly.

NASA Space Place, “Why is the sky blue?” — a technical explanation grounding the example, not evidence about understanding or cognition.

The exam is administered carefully#

No notes, no visible device, no source in the room. You ask Farhan to explain the daytime sky first, then the sunset, then you raise one objection of your own, live. This matches the resource-control and real-second-person standard the course has been arguing for since Week 6 and Week 11.

Nothing about the administration is careless. That matters for what comes later.

Rung 1 and 2: recognition and recall#

With the NASA-style explanation briefly shown, Farhan repeats it accurately — Recognition. Notes removed, he retrieves the same content unprompted a few minutes later — Recall: “Blue light scatters more because it’s a shorter wave, and that happens in all directions, all over the sky.”

Both pass cleanly. Neither is surprising yet — Week 5 already showed these two rungs are the weakest kind of evidence.

Rung 3 and 4: reconstruction and explanation#

Asked to rebuild the whole mechanism from nothing, Farhan does: “Sunlight hits the air, the gas molecules scatter the short blue wavelengths much more than the long red ones, and that scattered blue is what reaches you from every direction — that’s why the sky looks blue instead of the sun looking blue and everything else looking normal.” Parts named, related, and the causal step stated — Reconstruction.

Asked to explain it to someone with no background, he recasts it with an analogy about light bouncing off tiny obstacles far more than it travels straight through — pitched for a listener, not just for himself. Explanation, cleanly.

Rung 5: justification#

You ask why shorter wavelengths would scatter more than longer ones. Farhan answers: “The NASA explanation I read said it’s because they’re shorter, smaller waves — I don’t know the physics much deeper than that, but that’s the stated reason, not just a fact I’m repeating.”

This is a defensible Justification: he is giving the reason inside the explanation, not just asserting the outcome — while being honest about the depth of his own grounding, which is itself relevant later.

Worked response: rung 6, the Challenge#

You put exactly that objection to Farhan live: “Isn’t it contradictory — blue reaches you because it scatters, but at sunset blue is lost because it scatters?”

Farhan answers without hesitating: “No — scattering happens the whole way, in every direction, the whole time. At midday the path through the air is short, so enough of that scattered blue still reaches you from all around. At sunset the path is much longer, so the blue gets scattered again and again before it ever gets to you — by the time it would reach your eyes, it’s already gone somewhere else, and what’s left is mostly red and yellow.”

Does that response hold up?#

It does real work: it doesn’t retreat from the original explanation, and it resolves the apparent contradiction using only the two facts already established (scattering happens in every direction; sunset light travels through more atmosphere) — nothing new is smuggled in. By Week 5’s standard, restating the original account more firmly would not have counted; this does more than that.

This is genuinely Challenge-rung evidence: a live, unrehearsed-looking response to an objection nobody scripted for him.

Rung 7: the Boundary, volunteered#

Without being asked, Farhan adds: “I should say — I don’t actually know why sunsets are sometimes more orange than red, or whether smoke or dust changes it. I’m confident in the scattering explanation itself, but not in the finer detail past that.”

Nobody asked him to name a limit. He found the edge of his own account and said so — exactly Week 5’s definition of Boundary: naming, unprompted, where your explanation stops working.

What the examiner learns afterward#

A friend of Farhan’s mentions, in passing, that the evening before the exam Farhan spent over an hour in conversation with an AI assistant — told only that “something about light and colour” would be on the exam — working through the daytime/sunset explanation, a rehearsed response to “isn’t that contradictory,” and a suggestion to admit, if pushed, that he didn’t know the finer detail past the basic scattering account.

Nothing about the exam’s administration was at fault. This information arrives after the fact, second-hand, and is itself imperfect evidence.

Why this reopens the case rather than closing it#

Week 5 named an assumption built into the ladder’s own wording: Recall and Reconstruction are defined by the source being absent at the moment of performance — nothing about either rung’s definition asks where the content itself came from before that moment.

A well-rehearsed answer, produced collaboratively with an AI the night before and retrieved from memory during the exam, satisfies “produced without the source in front of me” exactly as well as an answer Farhan worked out entirely alone. The ladder was never built to tell those two histories apart — because Recall and Reconstruction only check for a source in the room, not for a collaborator in the past.

Two verdicts, both defensible, neither final#

Verdict A: count the pass. The exam was administered correctly, on the declared standard the course has argued for since Week 11. Some preparation before an exam is normal and expected; you cannot penalise Farhan for having studied the announced topic. Record the pass, with the preparation condition noted honestly.

Verdict B: mark it inconclusive. The specific objection Farhan handled so cleanly may be one he had already rehearsed an answer to, which would make it far weaker evidence of live reasoning than it appeared. Without a topic or objection Farhan could not have anticipated, this exam cannot rule that out.

Useful evidence is not the same as sufficient evidence#

Farhan’s performance is useful evidence of something real: under live questioning, from a real examiner, he can produce a structurally complete, accurate, well-organised account of this topic and respond coherently when it is challenged. That is not nothing.

It is not sufficient evidence that this understanding is independently his, because the identical transcript is equally consistent with a well-prepared recollection of material built together with a collaborator. The instrument cannot, from the transcript alone, tell you which explanation is true.

Two ways this could mislead you#

A false positive: concluding “Farhan understands” when the performance was substantially a rehearsed, AI-co-produced script, recalled convincingly under pressure.

A false negative, worth naming as the flip side: someone who understands the material deeply but freezes, rushes, or explains badly under live questioning could score lower on this same exam than Farhan did — the instrument can also undersell a real understanding it had every chance to detect.

Why one more rung will not fix this#

It is tempting to propose “Rung 8: explain something you could not possibly have rehearsed.” That only moves the problem, it does not solve it. However carefully you pick a genuinely novel topic, you are back to trusting that no collaborator helped Farhan with it either — live, off-camera, or through preparation you have no way to inspect.

Every rung the course has built converts an observed performance into a claim about what produced it. No rung — and no number of additional rungs — can make that conversion certain, because the thing being inferred is never the thing being observed.

Worked response: one way to write a defensible claim#

“My Week 11 examination shows that, under live questioning with sources removed, I could explain [topic] and respond to one unforeseen objection. It does not show that I could do this on a topic I had no chance to prepare for, or that my Challenge-rung response was not partly something I had already rehearsed for a different reason.”

Notice what this claim does: it names the exact condition the evidence was produced under, and it names the exact thing that condition cannot rule out. Neither sentence is a hedge added out of caution — each is doing real work.

What a disciplined claim is not#

It is not “I’m not completely sure, but I think I understand it” — that hedge does not say what is uncertain or why. A useful limit names the specific gap the evidence leaves, the way Week 1’s evidence record first asked you to.

It is also not silence. Refusing to conclude anything from a strong performance because it might have been rehearsed throws away real evidence along with the uncertain part of it. Verdict A is not the wrong verdict — it is one defensible reading, held alongside Verdict B, not instead of it.

Read further: claims, evidence and their limits#

Booth, Colomb, Williams, Bizup and FitzGerald. The Craft of Research (University of Chicago Press), Part 3, “Making an Argument” — the parts on warrants and on acknowledging and responding to objections. This book is used here for its general framework: a claim needs a stated warrant and an honest qualification of what it does and does not establish, which is exactly Verdict A and B’s shape above.

Rozenblit & Keil (2002), reused a final time. Their diagnostic-question method — asking someone to explain past the point where fluency runs out — is one concrete way to generate a Challenge like the one Farhan faced.

The Craft of Research is cited here at the level of its public table of contents (Part 3’s structure), not from a page-level reading of the text. No page number or direct quotation is claimed.

Sources and your next step#

Technical background: NASA Space Place: Why is the sky blue?

Farhan, the exam and the overheard conversation are fictional teaching material, built to test the course’s own instrument, not a report of a real case.

For the complete written activity and a record to keep, open the Week 12 investigation. The lecture notes offer a continuous reading version.