The Experiment That Lied
Alex’s dashboard has picked a winner. The spreadsheet would like a word.
Predict, inspect, revise. Follow one tempting conclusion from the Week 4 exposure dashboard to the evidence note that will accompany Alex’s fictional release.
Investigate on paper
The interactive investigation needs JavaScript. A has 26 positive responses in 100 exposures; B has 19 in 100. Predict whether that establishes a better bio, then open the evidence.
Open the four exposure cells
- North, A: 24/80 = 30%.
- North, B: 7/20 = 35%.
- South, A: 2/20 = 10%.
- South, B: 12/80 = 15%.
B leads in both zones. A has more exposure in the higher-response zone. Neither summary establishes a text effect. The CSV does not identify Alex’s drafts or document random allocation. Write a cautious conclusion and a proposed trial with a frozen photo, random allocation within zones and a fixed stopping rule.
Keep that note for the Release Day paper exercise.
- 1. Predict
- 2. Pool
- 3. Inspect
- 4. Record
THE CLAIM ON THE WHITEBOARD
“Bio A wins. Ship the rewrite.”
Alex’s fictional drafts are ready. A separate synthetic dashboard seems to have picked a winner. The letters look reassuringly familiar.
Your job: decide what this evidence can support before that sentence reaches the release notes.
The dashboard has a winner.
Positive responses / exposures, pooled across zones. All counts are fabricated teaching data.
A leads by 7 percentage points. Has the dashboard answered Alex’s question?
THE UNOPENED SPREADSHEET
The denominator is waiting.
Make a prediction, inspect the totals, then open the rows that the headline left out.
The useful outcome is an evidence note you can defend eight weeks later.
The winner loses twice.
| Zone | A | B |
|---|---|---|
| North | 24/80 30% | 7/20 35% |
| South | 2/20 10% | 12/80 15% |
B leads by 5 percentage points in each zone. North supplies 80% of A’s exposures and 20% of B’s. Pooling compares different mixes.
Compare a shared 50/50 mix
Give each zone half the weight: A = 0.5 × 30% + 0.5 × 10% = 20%; B = 0.5 × 35% + 0.5 × 15% = 25%. This is a declared hypothetical target mix. It does not recover a causal text effect.
Still missing: individual assignment records and a link from these A/B labels to Alex’s drafts. A photo change is also unobserved; the CSV has no photo field.
THE NOTE THAT TRAVELS TO WEEK 12
What will you put on the record?
Keep observations, unknowns and a proposed follow-up separate. You can export an imperfect draft and revise it later; the feedback will identify its gaps.
The totals are a descriptive result. “We don’t know yet” is a defensible causal conclusion, and it leaves a useful next experiment to specify.
The JSON download and continuation link contain only this fictional fixture and your selected choices. Save the file for another session. The note is practice evidence, not a submitted assignment.
Take the evidence back to the course
Use this investigation during the existing reversal block in the Week 4 tutorial. Keep the note with your Bio experiment protocol and use its limitations in the data report. The full protocol still needs your declared outcome, window and analysis plan.
The balanced A/B calculator explores a separate fixture. For this reversal’s arithmetic and its limits, return to the Week 4 notes. Release Day revisits your conclusion in Week 12.