Week 4: Evidence changes with conditions

Read one fictional person's three accounts of the same lock, produced under three different conditions, and find out why intuitive judgements of understanding stop agreeing with each other.

15 March 2027

Open the slides

Retrieve your Week 3 record · 2 minutes#

Bring back your annotated reconstruction and retained judgement from Week 3. Keep the original unchanged. After reading Rowan’s three responses, write two sentences: which gap in your reconstruction would each condition reveal or conceal, and does that change your original judgement? If you have no record, use Priya’s worked reconstruction from Week 3, or write a short source-closed reconstruction from this week’s lock explanation before proceeding.

Why does the same person look more or less knowledgeable under different conditions?

This is a self-guided lesson. These notes and the slide deck each contain the explanations you need; no presenter is assumed. Allow about 40 minutes, with another 25 if you complete the Week 4 investigation. Use paper or a text document for your answers. You need no prior knowledge of locks, no lock, no tools and no paid resource of any kind. Rowan and all three responses below are fictional teaching examples written for this lesson; they are not transcripts and not research data.

What you will learn#

By the end you should be able to say what a condition is and why it is part of the evidence rather than a distortion of it; construct the strongest case on both sides of a genuinely ambiguous performance; explain why a prediction about a modified mechanism is unusually strong evidence and unusually fragile evidence at the same time; and state precisely what is missing when two careful examiners disagree and neither can show the other where they went wrong.

1. Where the course has got to#

Week 1 separated an observation (“Sam repeated the distinction while the explanation was visible”) from an inference (“Sam understands it”), made the examiner the person responsible for that step, and argued, with Clark and Chalmers, that using a reference does not automatically disqualify a performance. Week 2 found that a fluent explanation can conceal missing relations. Week 3 took the source away, had its subject reconstruct the material and then annotate what was lost, and found that reconstruction shows more but can still be rehearsed or assisted.

All three weeks examined one performance at a time. Each ended by wanting more evidence. This week supplies more evidence — three performances by one person on one concept — and the difficulty that emerges is not a shortage of evidence at all. It is that the three pieces do not agree, and that there is no obvious principle for deciding which of them to believe.

2. The lock, explained from scratch#

Week 3 used a pin tumbler lock and this week uses the same lock on purpose. The whole demonstration depends on the object staying fixed while the conditions move, so changing it would destroy the comparison. The retrieval activity above uses your Week 3 record if available; its fallback and the explanation here make the lesson self-contained.

A pin tumbler lock is the kind in most front doors. It has a fixed outer casing, and inside the casing sits a cylinder that can rotate, called the plug. The key goes into a slot at the front of the plug called the keyway. At the back of the plug is a cam: a small arm that pulls the bolt out of the door frame when the plug turns. So the key has exactly one job to accomplish — making it possible for the plug to rotate.

Sitting across the boundary between plug and casing is a row of small stacks, usually five or six of them. Each stack has a key pin at the bottom, resting in the plug; a driver pin above it; and a spring above that, pressing the whole stack downward. The shear line is the boundary itself: the circular join where the outside surface of the plug meets the casing around it. It is not a component you could pick up. It is the line along which the plug has to be able to turn.

With no key in the lock, the springs push every stack down, so each driver pin drops out of the casing and part-way into the plug. Pins now straddle the shear line, pinning the plug to the casing so that it cannot rotate. The key pins come in different lengths, and the cuts along the key are different depths to match. A correctly cut key lifts each key pin by exactly the amount that puts the junction between that key pin and its driver pin level with the shear line. When every junction is level with it, nothing crosses the boundary; the plug turns, the cam moves, and the bolt withdraws.

A wrong key still lifts the stacks — it lifts them by the wrong amounts. Lift a stack too little and its driver pin is still hanging down inside the plug. Lift it too far and the key pin itself is pushed up into the casing. Either way something crosses the shear line and the plug will not turn. Notice the consequence, because it matters later: the lock does not grade you. There is no partly open. Anything crossing that line, anywhere along the row, is enough.

Technical grounding: Wikipedia, “Pin tumbler lock”. That article supports the mechanism and nothing else. Under this course’s source rules, technical documentation grounds a worked example but is not evidence for any claim about understanding, learning or cognition. This course has no interest in defeating locks and gives no method for doing so.

3. What a condition is#

A condition is a feature of the situation an answer was produced in, which could itself change the answer. What was available to consult. Who the answer was for. How much warning there was. What kind of question was asked at all.

The tempting picture is that a condition makes the same task easier or harder, so that a performance under a hard condition is the same evidence with more noise in it. That picture is wrong, and abandoning it is the point of the week. Removing a reference does not strip away assistance to reveal the underlying person; it adds a demand on memory, which is a different thing to have evidence about. Asking for a prediction rather than an explanation does not raise the difficulty of one scale; it asks for a different kind of performance. The condition helps produce the evidence. That is why the same person can look more or less knowledgeable without anything about the person changing.

4. Rowan, three times#

Rowan is fictional. They are given the same mechanism three times in one afternoon, under three conditions.

Condition (a): the reference visible#

Rowan has the reference text open on the table — everything in section 2 above. The question is: explain how a pin tumbler lock works. Nothing is timed and nothing is taken away.

A pin tumbler lock has a plug that rotates inside a fixed casing. The key goes into the keyway at the front of the plug. Along the boundary between the two there are five or six stacks of pins, and each stack is a key pin at the bottom, sitting in the plug, a driver pin above it, and a spring above that pressing down.

That boundary is called the shear line. With no key in, the springs push the stacks down so the driver pins cross the shear line into the plug, and that is what stops it turning.

The key pins are different lengths, and the key is cut to match them. The correct key lifts each key pin so that the junction between the key pin and the driver pin ends up exactly at the shear line. Then nothing is crossing the boundary, the plug turns, and the cam at the back withdraws the bolt. A wrong key lifts the pins by the wrong amounts, so something still crosses the shear line and it stays locked.

Condition (b): from memory, to a novice#

The reference is closed and removed. The listener is a friend who has never once wondered what happens inside a key. The question is: explain to them how a key opens a lock. Rowan now has two jobs at once — recover the mechanism without the text, and recast it for someone with no vocabulary for it.

OK, so — you know the zigzag notches along the top edge of a key? They are not decoration. They are a code, and the lock is the thing that reads it.

Inside the lock there is a barrel that has to spin round for the door to open. Normally it cannot, because there is a row of little pins, five or six of them, sticking down through the join between the barrel and the metal around it. A pin sitting across a join is a bolt. Nothing is turning.

Every pin is a different length and every notch is a different depth. The right key lifts each pin by exactly its own amount, so all of them come flush with that join at the same moment. Nothing is sticking across it any more, the barrel spins, and a little arm at the back pulls the bolt out of the frame.

A wrong key does lift the pins. It just lifts them to the wrong heights, so one is always left poking across the join, and it stays shut.

Condition (c): a prediction about a changed case#

Still nothing to consult. The question is one Rowan has not been asked before: take a key that works this lock; one single cut on it, say the third one along, has been filed a little too shallow, and every other cut is correct. What happens when you put that key in and turn it, and why?

Sorry — do you want me to guess, or is there a right answer? … Right. OK.

Too shallow means it does not lift that pin far enough. So that one is sitting low. The bit of it that is supposed to end up level with the join is below the join instead, so something is still crossing, at that one position.

And — this is the bit I am least sure about — I think it does not matter that the other five are fine. One is enough. Because the barrel is one piece. It is not five separate locks that each open a fifth of the way. So it does not half turn or feel loose. It just does not turn, the same as a completely wrong key.

Hold on. Would you even be able to tell this key from a totally wrong one, from the outside? I do not think you would. That feels wrong to me but I cannot see what would make it false.

Rowan cannot have read that answer anywhere, because the key does not exist. Measured against the mechanism in section 2, it is also correct.

5. Pause and write: judge all three#

Allow five minutes. You are the examiner. Write one sentence about each response saying what it gives you evidence of, then one sentence answering: does Rowan understand how this lock works? Use section 2 freely — this is not a memory task. Record which response you found most convincing while it is still fresh. Attempt all four sentences before reading on.

There is a sharper version of the sceptical objection, and it is worth having because it attacks the strongest plank of the other case. Rowan read the reference under condition (a), and that reference says anything crossing the shear line anywhere along the row is enough to stop the plug. So the headline conclusion of (c) — one bad stack is sufficient — was already available in print, and the claim that (c) “could not have been memorised” is too quick. What the reference does not supply is Rowan’s reason: that the plug is one object, so there is no partial turn and no way to feel the difference from outside. That inference, and the question Rowan closes with, are the parts of (c) that have no source. If you want to defend condition (c) as the strongest evidence, defend those and not the conclusion.

Neither of those readings is a misreading. Each is supported by things Rowan actually said and each ignores nothing. That is the uncomfortable part: if you now go back and look harder at Rowan, you will not settle it, because there is no further detail in these three responses that decides between the two cases. The disagreement is not about Rowan. It is about which condition you let count.

6. Evidence is produced by a situation#

This is the week’s central idea and it is worth stating plainly. A performance is not a window onto a person, clearer or cloudier depending on how hard you look through it. The condition participates in producing the performance. Take the reference away and you do not get the same evidence with the assistance subtracted; you get different evidence.

One immediate consequence is that “does Rowan understand the lock?” is underspecified as a question. Understand it well enough to do what, under which conditions? Week 1 asked you to record the conditions an answer was produced under. This week explains why that record is not bureaucracy: without it, the claim has no content.

7. Why condition (c) is both the sharpest and the most fragile#

A prediction about a modified mechanism is the only one of the three conditions that cannot be satisfied by holding the parts in the right order. To answer it you have to know what each part does to the next: that cut depth sets pin height, that pin height decides whether something crosses the shear line, and that the plug is a single object so that one crossing is decisive. Part names will not produce that answer; causal relations will. That is a real advantage and it is why (c) carries the most.

Now look again at how Rowan opened it: “do you want me to guess, or is there a right answer?” Somebody can hold the entire mechanism and still stall on (c), because they have never been asked to predict anything before and do not know what kind of answer is wanted, how much detail counts, or whether hedging is permitted. The genre of the task is itself unfamiliar. So a weak answer under (c) is ambiguous between two completely different explanations — no causal model, or no experience of being asked — and the transcript alone will not separate them.

That is the double edge, and it is exactly why intuition becomes inconsistent. The condition that produces the most informative evidence when it works is also the condition most easily disturbed by something with nothing to do with locks. An examiner who does not notice this will read hesitancy as ignorance in one person and as care in another, without ever having stated a rule that could be challenged.

8. Pause and write: which condition do you trust?#

Allow three minutes. Write which of the three conditions you would trust most and one sentence of reasoning, then a second sentence naming the strongest objection someone could make to your choice. You are not being asked for the right answer; you are being asked to make your reason visible enough that somebody could disagree with it.

9. One thing worth noticing across all three#

Rowan is vague about the same thing twice. In (b) it is one pin coming “flush with the join.” In (c) it is “the bit of it that is supposed to end up level with the join.” Both times the two-pin stack has collapsed into a single pin.

That repeated gap is more informative than either response by itself. A gap that appears under one condition might belong to the condition — a slip of phrasing for a novice, a hesitation under an unfamiliar question. A gap that survives a change of condition is much harder to blame on the situation. This is the first piece of genuine leverage the week offers, and it comes from comparing performances rather than from examining any one of them more carefully.

It is also, honestly, a small piece of leverage. Two observations are not many, and the two conditions that produced them shared the absence of the reference, so they are not independent.

10. If your judgements were consistent#

Some readers give the same verdict under all three conditions and feel no tension at all. That is a real result and it deserves to be taken seriously rather than treated as a failure to notice something. The curriculum’s “limitation to investigate” is not a requirement that you personally waver.

But be precise about what consistency establishes. It shows that you applied something stable across the three responses. It does not show that the something was a good criterion, and it does not show that you could state it if asked. The most common version is that you formed a view under one condition and read the other two through it, without ever registering that you had chosen a condition. A useful check: write down what evidence would have changed your verdict. If nothing would have, your consistency came from a commitment rather than from the evidence.

11. What this week leaves you with#

Held against three conditions, intuitive judgement is not stable. It moves with the situation and it moves without announcing that it has. That is not a reason to distrust intuition entirely — it produced real observations here: the missing second pin, the hedged opening, the independently phrased relation. The problem is not the observing. It is the weighing.

At no point today could we say, in advance and out loud, what would make one of Rowan’s responses stronger evidence than another. Without that, a disagreement between two careful examiners collapses into each asserting which condition mattered, with no way for either to be shown wrong. And every judgement is unrepeatable, including your own next week on the same three texts.

What is needed is an instrument: a rule, written down before you look, saying what counts as stronger and weaker evidence, so that it can be inspected, applied by someone else, and argued with. Week 5 builds the course’s first one — a ladder of performances running from recognising something up to connecting it to other things, with each rung defined well enough that two people can apply it and locate their disagreement. This week deliberately stops at naming the need. Do not write a ladder yet; the point of Week 5 is partly that constructing one forces choices you cannot see from here.

12. Keep a short record#

Before you finish, write four lines about your own judging and keep them:

The third line is the one that matters and the one most likely to be hard to write. If it comes out as “it just seemed more convincing,” that is the finding, not a failure. You have located the thing Week 5 has to supply.

Reading with a purpose#

Primary empirical source. Rozenblit, L., and Keil, F. (2002). “The misunderstood limits of folk science: an illusion of explanatory depth.” Cognitive Science, 26(5), 521–562. Full text from the authors’ lab. Read the Study 1 procedure. Participants rated their own understanding of everyday mechanisms on a 1 to 7 scale; ratings fell after they wrote a step-by-step causal explanation, and again after answering a diagnostic question about the mechanism. There were eight test items across two sets, with four explained by each participant; one set included a cylinder lock. Ask: how much of that movement belongs to the change of task rather than to the person?

Three cautions, because they are the week’s own point. Study 1 was 16 graduate students at one university. These are self-ratings — self-report, not independent evidence of what anyone could produce — which is why we say the study echoes this week rather than demonstrating it; our week concerns an outside examiner’s judgement, and that extension is ours, not theirs. And the study’s own comparison between the explanation condition and the diagnostic condition was not statistically significant, which is a useful corrective: the condition matters, but which conditions matter is an empirical question and not something intuition can tell you in advance. Their diagnostic question for the lock concerned defeating it; ours deliberately does not, and this course gives no method for it.

Theoretical argument. Clark, A., and Chalmers, D. (1998). “The Extended Mind.” Analysis, 58(1), 7–19. Read the authors’ text, sections 1 to 3. Week 1 used this to argue that a reference does not automatically disqualify a performance. Ask the harder version this week needs: what would have to be true of Rowan’s use of the reference under condition (a) for it to count as part of the thinking rather than a substitute for it — and could you tell, from the transcript alone? This is a philosophical argument, not empirical evidence.

Optional companion. Brown, P. C., Roediger, H. L. III, and McDaniel, M. A. (2014). Make It Stick: The Science of Successful Learning. Belknap Press of Harvard University Press, chapter 1, if you have access. Author resources. Week 1 used it for the unreliability of ease as a signal to the learner. Ask the question this week raises: is ease any more reliable as a signal to the examiner? Buying the book is not required to complete this lecture.

What this leaves unresolved#

Three performances by one person under three conditions do not combine into a verdict, because nothing in the evidence says how to weigh them. Two careful readers can reach opposite conclusions from the same texts and neither can show the other where they went wrong. Apply this yourself in the Week 4 investigation, where you judge the three responses in writing and then examine your own judging. Next week we stop relying on intuition and build a stated instrument — and then spend the weeks after it attacking the assumptions that instrument quietly makes.