Week 10 Session: Designing a control experiment

Drafting one experiment protocol and pressure-testing it against five fixed questions, before the Waiting Experiment asks you to run it.

Week 10’s lecture named two easy mistakes: turning a control experiment into an information experiment by accident, and using crowding — this week’s own source’s setting — as if it were the thing to manipulate. This session exists to catch both, in your own design, before the assessment does.

Before the session#

Read the Week 10 lecture first. Bring a draft protocol, even a rough one: your starting point, what you’d manipulate, what you’d hold constant, what you’d measure, and your prediction.

Choosing a starting point#

Two legitimate ways in, matching the Waiting Experiment brief:

1. Week 10’s mechanism: control. Vary the availability of a genuine, no-penalty choice to continue or leave a wait, while holding wait-time and status information constant across conditions.
2. A named gap. A mechanism your Queue Case Study marked ambiguous or undeterminable, or a specific, statable limitation of a course source — named exactly, not gestured at.

Nobody is required to test control. What everyone is required to do is name, precisely, why their chosen mechanism is the right one to test.

The five questions#

Bring your protocol to the room. For every design in the session, including your own, answer these in order:

1. Is there exactly one manipulated variable? If two things differ between conditions, the design tests neither cleanly.
2. If you’re testing control, is control actually what’s varying? A design that changes what information a participant receives, and then measures “control,” has tested information — however it’s labelled.
3. Are the other mechanisms held constant, as far as reasonably possible? Name what you’re holding fixed — the wait-time and status information, the environment, the queue structure, the priority class, the value on offer — not just what you’re varying.
4. Does your measure actually match your manipulation? A design that varies choice but measures judged duration is measuring the wrong thing unless duration is specifically what you predicted would move.
5. Does the design satisfy the experiment-ethics policy? Self-observation, a simulation, or a controlled prototype by default. No live intervention on a real, non-consenting person or a real public queue.

A design that fails any of these gets revised in the room, not carried forward as-is.

A worked example to test the questions against#

A controlled prototype, run with a few consenting participants: a mock waiting screen shows identical wait-time and status information in both conditions. In condition A, participants are told they must wait until the process finishes. In condition B, participants are told, up front, they may continue or leave at any point with no penalty. This is a course-designed simulation, not Hui and Bateson’s own procedure, and it doesn’t test judged duration — the measure is a short self-report of perceived control, and/or whether a participant given the option actually leaves. Note honestly, in your own write-up if you use a design like this, that telling someone a choice exists is itself information; what’s held constant is the wait-time and status information, and the variable under test is the availability of the choice itself.

What’s out of bounds#

No design that requires a live intervention on a real queue or on non-consenting people. No design that quietly manipulates crowding and calls it control. No prediction written or revised after seeing a result.

What this can and can’t show#

A protocol checked in this room is a better-designed protocol — it isn’t a run experiment. Nothing here tells you what your actual result will be, and nothing here turns a small, largely uncontrolled exercise into a controlled study.

Afterwards#

Take your checked protocol into the Waiting Experiment, due at the end of this week: prediction, protocol, manipulation, measure, observation, bounded interpretation, limitations, in that order. Whatever you find, the Waiting Audit in Week 12 will ask you to state whether that result is relevant to the system you’re auditing, name exactly what it tested, and tier it honestly — a mechanism’s classification and the strength of its evidence are two separate questions there, and keeping them separate starts here.