Week 11 Session: Design and administer a real examination
Design a short instrument from Week 11's five decisions, then administer it to a real second person and keep an honest record of what it does and does not establish.
3 May 2027
Where Week 10 left us#
Week 10 showed why a performance with a collaborator cannot automatically be attributed to one person. This week turns that limit into an examination decision and retrieves Week 9’s prior-state record. We reuse Week 5’s seven rungs: Recognition, Recall, Reconstruction, Explanation, Justification, Challenge and Boundary. They classify responses under stated conditions; they do not certify understanding.
Purpose and preparation#
This self-guided activity applies Week 11’s lesson. Allow about 20 minutes to design your instrument and roughly another 20 to 30 minutes to administer it and record the result — you will need to arrange that second block of time with another person, so read this whole page before you start.
This week’s core task requires a real second person. A solo rehearsal is offered below as a fallback, but read section 5 before choosing it: it states plainly what it loses, and that loss is not small. You are examining another person’s understanding, not completing a technical certification or a graded test of them — choose a topic they will not find punitive to discuss, and ask them first.
1. Choose a topic and a person — 5 minutes#
Pick someone you can realistically talk with for 20 to 30 minutes: a housemate, a friend, a family member, a classmate. Pick a topic they genuinely have some standing to know — a hobby, a skill from work, something they have actually practised — not a topic from this course, and not one chosen because you suspect a weak spot.
Ask them directly: “Can I talk to you for about 20 minutes about [topic]? I’m doing a course exercise about how to find out what someone actually understands, not just what they can say.” Getting a plain yes matters; this is not a task to spring on someone unannounced.
2. Make your five decisions — 5 minutes#
Write down, before the conversation happens:
- Topic and person — who, and what, and why they have standing to know it.
- Resources — will they have notes, a phone, or a reference available during the conversation, or not? State the one-sentence reason, tied to what you actually want to find out.
- Notice — what will you tell them in advance? Full notice of every question, partial notice of the topic only, or none? State the reason.
- Target rungs — write three or four tasks (questions or short prompts), each one labelled with the rung from Week 5’s ladder it is aimed at: Recognition, Recall, Reconstruction, Explanation, Justification, Challenge, or Boundary.
- Live collaborators — whether another person or AI may contribute during the examination, and why.
Keep this written record; you will compare it with what actually happens.
Collaborator decision and prior-state record#
Decision 5: live collaborators. State whether another person or AI may contribute while the subject answers, and why. Wren excludes live collaborators to examine Petra’s own responses; an assisted version would require a record of who contributed what and a claim about the joint performance.
Before the first task, ask the subject to record one claim about the topic, a reason and their confidence, before consulting anything or receiving feedback. Keep it unchanged beside the final answer. This Week 9 prior state establishes what was recorded at that point, not everything they previously knew.
For example, a fictional prior note might say: “A finer grind needs less extraction time because water contacts more surface; confidence: medium.” Later agreement shows that the claim was recorded before the tasks; it does not prove that every later justification was already available.
3. Administer it — 15 to 20 minutes#
Have the conversation. Ask your prepared tasks roughly in order, but let a genuine follow-up happen if an answer raises one — Week 11’s worked example found real Boundary-rung evidence from an unplanned follow-up, not only from the four prepared tasks. Do not read from a script rigidly if it makes the conversation stiff; the tasks are a plan, not a transcript to recite.
Write down what your subject actually says, as close to their own words as you can manage, during or immediately after the conversation. Do not tidy or improve their answers afterward — a record you have edited toward your own expectation is no longer evidence of what they said.
4. Classify and interpret — 5 to 10 minutes#
For each answer, write down which rung it reached and the specific words that earned it. Some may land on a rung you did not aim a task at; record that honestly rather than forcing it onto your original plan.
Compare with Week 11's worked example
Wren’s four tasks aimed at Recall, Reconstruction, Justification and Boundary. Petra’s answers reached all four, including a Boundary-rung answer (“humid days… I couldn’t tell you the mechanism”) that Wren had not predicted the content of. The record still could not establish whether Petra’s Justification answer existed in that form before being asked, or was assembled live — no single conversation settles that question, however well the instrument is designed.
Then write, in two or three sentences, what this record does not establish — a specific limit, not a general hedge. “They might be wrong” is not specific enough; “this does not establish whether their explanation existed before I asked, only that they could produce it when asked” is.
5. Solo route, and what it loses#
If you genuinely cannot arrange a second person, you may design an instrument and answer your own questions as a rehearsal, keeping the same five-decision structure. Do this only as a fallback, and record explicitly that it is one.
State plainly what this loses: you already know every answer before you ask the question, so nothing about your own performance can be unanticipated in the way Petra’s Boundary answer was unanticipated by Wren. You will practise the design skill from sections 2 and 3 of Week 11’s lesson, but you will not produce genuine examination evidence, because an examination requires an independent second mind producing something the examiner could not have written themselves. Record this limitation in your own words rather than skipping past it.
6. Keep a short record#
Keep, in a document or on paper:
| Field | What to write |
|---|---|
| Prior state and collaborators | The original claim, reason and confidence; the collaborator policy and any contributions actually made. |
| Decisions | Your topic, subject, resources decision, notice decision, and target rungs, as written before the conversation. |
| Transcript notes | What your subject actually said, close to their words, for each task and any real follow-up. |
| Rungs reached | Which rung each answer reached, with the specific evidence. |
| Surprise | Any answer that reached a rung you had not aimed a task at. |
| Remaining uncertainty | What this record specifically does not establish. |
| Route taken | Real administration, or solo rehearsal — and if solo, what section 5 says was lost. |
This record is what Week 12 asks you to bring back and interpret against a harder question: what are you actually justified in concluding from it?
The spec
What you bring to the session. Written so a reader can tell whether it has been met without asking you.
- design an instrument with a stated resources decision, notice decision, and one task per targeted rung
- administer the instrument to a real second person and record their answers as given
- classify each answer on the Week 5 ladder and name at least one thing the record does not establish