Week 11: Post-Date Analytics and The "Second Date" Metric

Separate observed agreement from inferred subtext. Audit the time budget, evaluate a frozen profile and prepare a reproducible release.

17 May 2027

Open the slides. Use A/D or scroll to turn pages; Esc returns here.

Alex’s story · One minute changes the dashboard#

Fictional course story · Week 11.

Alex opens the follow-up report. At the exercise cutoff, only one next-date agreement is visible; the completed authored trace contains both. Meanwhile, the frozen profile’s benchmark result is rank 2/100. Moving the cutoff or the scoring weights could make a tidier headline. Alex’s release now needs to show which result answers which question, including the result that missed its target.

Investigate this week: Reconstruct the cutoff label and conservative rank. Can another reader reproduce the release while distinguishing baseline, sensitivity and changed-case results?

The dashboard after the date#

The dashboard has four labels. None of them is “probably into you.” This week builds a Follow-up evaluation: observation rules, a time-allocation critique, a frozen benchmark result, two separate sensitivity cases and a reproducible static release candidate. A disappointing result belongs in the record.

By the end, you should be able to reconstruct a label at a cutoff, defend a denominator, check the units of an ROI claim, reproduce a conservative rank, distinguish changing weights from changing evidence, and verify that the released text is the text evaluated. All records, values, ratings and probabilities are authored fictional teaching data. No personal experiment or private conversation is required.

Scheduled teaching and independent work#

60-minute lecture#

Monday, 11 am–12 pm. Use only these selections from the complete deck, in order. The timings include explanation, brief working and questions; full exercise timers elsewhere belong to the complete resource route.

90-minute tutorial#

Thursday, 2–3.30 pm. Follow the Week 11 tutorial: 20 minutes log audit; 25 minutes frozen evaluation; 20 minutes changed conditions; 25 minutes release review and quiz. The last block includes ten minutes for the interactive quiz and its debrief. These four blocks fill the whole session. Bring lecture working for checking and refinement; complete each exercise once. Solo work and review are accepted.

Independent work · about 210 minutes#

• 30 minutes — before Monday: Bring the exact Week 5 freeze, Date transition model and Offline handover plan. Inspect their versions before evaluating anything.
• 90 minutes — between Monday and Thursday: Review remaining worked examples, especially cost overlap, candidate evidence and changed-case tests (slides 14, 25–26 and 36–42). Download the sixteen-file workbook, locate your own release’s build instructions and generate the labelled worked-example release into a new directory for Thursday’s browser and reproduction review. Preserve your own frozen candidate for evaluation before revision.
• 90 minutes — consolidation and assessment: Finish the Follow-up evaluation and inspect the generated HTML in a browser. Preserve the failed target, exact files and reproduction instructions for next week and the project.

Together this is about six hours: 60 + 90 + 210 minutes. Slides and full-length exercises outside the lecture selection are independent study unless assigned in the tutorial’s four blocks. Use the worked answers to check an attempt; optional extensions can use consolidation time. The complete route below reuses this material and adds no scheduled meeting or extra three-hour study requirement.

Complete teaching pack#

The full 50-slide pack and all worked notes remain the complete resource. This 180-minute reference route includes a ten-minute break for someone facilitating the whole pack in one sitting. It is separate from the scheduled selections above: the break slide is skipped in the 60-minute lecture and the tutorial keeps its own 90-minute plan. These are facilitation estimates, including practical work and debriefs.

These are facilitation estimates, not a duration guarantee from slide count. Faster groups can spend another 15–20 minutes deriving the weight crossover, challenging a rating with evidence, or comparing two clean release directories. Extensions add no assessment requirement.

Bring the evidence, including its boundaries#

Bring the Week 5 frozen candidate and objective, Week 9 Offline handover plan and Week 10 Date transition model. The plan supplies its version and agreed costs; the meeting exporter supplies coded events, bill status and the fact that the meeting ended. Its output does not turn a simulation into observations.

The workbook actually replays the earlier conversation, authored threat clearance, confirmed handover and meeting events through the existing functions. Clearance remains an assumed review decision, not identity proof. Alternative meeting branches are different versions of one fictional scenario; do not count them as independent dates. Candidate and invitation references remain in the report alongside the Week 9 cost agreement.

Week 10’s exact two-step vector is still (0.31, 0.16, 0.32, 0.21), with seeded 10,000-path counts (3108, 1606, 3182, 2104). Neither 0.32 nor 3182 establishes an actual agreement. Keep that comparison in the existing validation appendix.

Define the second-date metric#

A first date supplies sparse observations, not a labelled training set. “I had a nice time” records words. It does not provide an explicit next-date response. If a clarification is appropriate and wanted, draft one optional question such as “Would you like another board-game meetup? Fine to pass.” No response is owed, and the workbook sends nothing.

Meeting status is a separate field. Meeting ended + next date agreed is coherent. A later withdrawal must remain visible even if an earlier snapshot said agreed. In Week 10’s bounded event exercise, a decline is terminal and the meeting stops accepting events when it ends. This workbook reconstructs snapshots from that retained history; it does not add a live post-meeting messaging system.

Exercise: the cutoff falls between two replies#

Allow 5 minutes. In the actual Week 10 authored trace, Alex agrees at minute 1085, the counterpart at 1086, and the meeting ends at 1090. The clock is minutes since midnight on the fictional 7 May scenario. Define the window as [0, 1086). Report the label at the cutoff and the latest label in the completed export.

Debrief. The cutoff view is pending, with only event next-a. An event exactly at 1086 falls outside the half-open window. The latest export is agreed, with meeting: ended. Keep both; a late observation must not silently rewrite a frozen reporting window. In the withdrawal branch, the label before 1087 is agreed, while before 1088 it is declined. The full event history stays attached.

An observation window is a reporting convention, not a recommended waiting time. Closing your own follow-up does not prove the other person’s refusal. Coding a personal closure would require its own field, as Week 7 already demonstrates.

Exercise: improve the percentage without improving anything#

Allow 4 minutes. A separate synthetic dashboard contains eight distinct records at a common rule of 48 hours after each meeting: two agreed, two declined, three pending and one unobserved. Calculate the agreement fraction among all records and among resolved records.

Debrief. The answers are 2/8 = 25% and 2/(2 + 2) = 50%. Both calculations are defined, but they answer different questions. Excluding the four unresolved records doubles the headline without adding an agreement. Report all four counts and the cutoff. An all-pending dashboard has an undefined resolved-only fraction, represented as null, rather than zero. These eight authored rows estimate no population rate, treatment effect or success probability.

Key concept: return on investment#

Calculating Return on Investment (ROI) of time spent vs. romantic viability is a critique of the objective, not an instruction to price another person. This course’s benefit/time proxy allocates Alex’s own time. The invented “benefit” does not measure romantic viability.

The separate time log contains 90 minutes together, 20 minutes of travel and five minutes composing a follow-up: 115 minutes. The two travel legs each take ten minutes, including waiting and walking. Composition time establishes neither sending nor delivery. This fixture is distinct from Week 9’s 90-minute itinerary and its 50-minute meeting. Markov steps have no minute duration.

Exercise — 4 minutes. A revision adds five minutes composing while on the return bus at [105, 110), but retains the full ten-minute return journey. Can you add both durations to report elapsed time? What if an interval is missing?

Debrief. Those activities overlap. Their effort durations can be separately described, but summing them double-counts elapsed time. This bounded workbook rejects overlapping intervals so the assumption is visible. Adjacent endpoints are accepted. Gaps are reported as unallocated time; they are not silently charged to the meeting. Duplicate IDs, negative endpoints and backwards intervals fail.

The number changes with the invented benefit#

10/115 ≈ 0.087 units/minute is defined arithmetic. It is about 5.217 units/hour, not 8.7%. A conventional net-return expression (benefit − cost) / cost would require benefit and cost in compatible units; subtracting minutes from arbitrary benefit units is invalid. An explicit conversion could make a different toy model, but its time price would be another assumption. Zero time produces an undefined rate (null), not infinite viability.

Exercise — 4 minutes. Halve the assumed benefit, then double it. Has the observation record changed? Would reducing the recorded time by omitting travel improve the situation?

Debrief. The proxy halves or doubles because its numerator was chosen. Omitting travel improves the reported ratio by changing the accounting boundary. Neither operation adds mutual interest. There is no justified universal threshold for continuation.

Past effort does not authorise the next action#

Reuse Week 7’s decision comparison: U(stop)=0, U(wait)=2q−1, and, only when separately admissible, U(clarify)=7q−5. The parameter q and the five-unit clarification cost are assumptions on Week 7’s utility scale. Five units are not the five composition minutes. Do not estimate q from our tiny dashboard, infer it from “nice time,” or substitute the Markov N probability.

Exercise — 5 minutes. Derive the stop/wait crossover and the wait/clarify crossover. At q=0.9, remove clarification from the admissible set. Then add an explicit refusal.

Debrief. 2q−1=0 gives q=0.5. 7q−5=2q−1 gives q=0.8. With clarification excluded at 0.9, wait is the numerical maximiser among the remaining actions. An explicit refusal terminates the invitation before numerical optimisation: stop. The generic comparison function’s admissibility flag is not a complete consent protocol. The earlier terminal state rules still apply. Subtracting the same already-incurred time cost from every future option would not change their ordering; it cannot create permission to continue.

Freeze before evaluating#

A candidate is more than four numbers. Save its exact text, case version, evidence quotes and reasons, ratings, invitation reference, controls version, scoring rule and conservative tie rule. Freeze those inputs before looking at alternatives. Store the primary result before exploring a revised bio.

The default workbook assembles a worked teaching snapshot now from Week 4’s existing supported repair and Week 5’s library invitation. It uses the earlier worked vector (4,4,3,4). It does not claim to recover a student’s previously frozen candidate or release. Use your own Week 5 freeze first; the example is a reproducible reference.

Board games, bad puns, maybe a relationship? Meet Friday 5-7 pm: bus-friendly, $20 total. Fine to pass.

This is the exact repair field in the unchanged Week 4 bio file. Alex’s original facts remain: adult CS student, board games and terrible puns, possible relationship, Friday 5–7 pm, bus travel and $20 outing budget. No new biography is added. The old photo-version reference is retained as provenance; this minimal example renders text only.

The rating anchors distinguish absent (0), vague (1), contradicted (2), usable but incomplete (3), and explicit/supported (4). The separate Week 5 library route takes 100 minutes and $14, fitting the original 120 minutes and $20; that notebook evidence does not magically insert missing venue details into the bio. A quote-presence check can catch a missing quote. It cannot decide whether a human rating is justified.

Primary score and the conservative tie rule#

Null Island v1 contains 99 unchanged synthetic controls plus one candidate = 100 entries. It is an authored comparator, not representative app data or a held-out user study.

score = 100 × (clarity + specificity + feasibility + exit) / 16
rank = 1 + count(control score >= candidate score)

For (4,4,3,4), the sum is 15 and the score is 93.75. No control scores higher, but one control ties: rank = 1 + 0 + 1 = 2/100. Only rank 1 receives the benchmark’s top-1% label. Ties count against the candidate, so a > comparison is wrong. Do not round scores before ranking.

Exercise — 6 minutes. Manually score control control-099=(3,4,4,4) and compare it with the worked candidate. A colleague reports “joint first, therefore top 1%.” Identify the failed contract.

Debrief. Both total 15 and score 93.75. The course uses a conservative rank, not a joint-first convention. The target fails. Rank is not a marking criterion; explaining this failed target can support an excellent evaluation. The ideal (4,4,4,4) scores 100 and ranks first in this deliberately constructed fixture, but ratings require evidence.

Change the weight for every entry#

Double feasibility’s weight without changing the candidate or any evidence:

weighted score = 100 × (clarity + specificity + 2×feasibility + exit) / 20

The maximum is 4+4+8+4=20. Recompute all 99 controls and the candidate under the same rule. Mixing a new candidate score with old control scores compares different scales.

Exercise — 7 minutes. Derive the probe’s 81.25 and 70, then reproduce its ranks from the CSV. Count greater scores and equal scores separately. Explain why the release’s unchanged rank does not imply an unchanged comparison.

Debrief. The probe has 13/16×100=81.25 and 14/20×100=70. Initially one control is above and three tie, giving 5. Under doubled feasibility, twelve are above and nine tie, giving 22. The release example moves from one tie to one strictly better control, so its rank stays 2 even as its score falls. That control now scores 95.

Optional extension: for weight w, the release numerator is 12+3w; control 099’s is 11+4w. With a common positive denominator, their difference changes sign at w=1. Pairwise ordering changes there; the conservative tie rule still counts equality against the candidate. Changing a weight expresses a different rubric preference, not a causal dating effect.

Change availability in a separate case#

Retain the original baseline and weight-sensitivity results. Now declare a different fictional case: Alex has only Friday 5–6:30 pm, or 90 minutes. This does not change Alex’s fixed course biography; it is a labelled stress test.

The unchanged library plan still needs 100 minutes, so its slack changes from +20 to −10 minutes. It fails the time constraint. The unchanged bio still promises Friday 5–7 pm, contradicting the changed case. Under the published anchors, this authored review changes feasibility from 3 to 2, leaving the other ratings unchanged. The primary score becomes 87.5, rank 2/100. Case fidelity fails, so release under the changed case is blocked regardless of rank.

Exercise — 6 minutes. List the statements and artefacts affected. Can the unchanged rank justify retaining the invitation? Does the 90-minute Week 9 itinerary rescue the 100-minute library plan?

Debrief. Review the time claim, library itinerary, dependent invitations and any agreement to those exact terms. A different venue’s duration cannot be substituted without changing the plan and seeking the relevant agreement. A new bio or shorter plan would be an exploratory revision with its own version and review. Do not silently repair the frozen candidate, mutate the fixed controls or overwrite the original result.

Controls are supplied numeric vectors without source profile text. We therefore keep them fixed in this candidate-only changed-case stress test. We cannot truthfully re-rate all controls for a new availability condition without their evidence. The doubled-weight comparison, by contrast, can and does rescore every entry. Neither exercise measures a real population distribution shift.

Evaluation can quietly become development#

scikit-learn’s leakage guidance explains why test information must not influence model choices and why learned preprocessing belongs on training data. Here the analogous mistake is editing a candidate after seeing its benchmark result, then calling the new result an untouched evaluation of the old candidate. This course benchmark is public and synthetic; even a careful freeze does not make it a hidden test set.

Exercise — 5 minutes. Classify these actions: (a) save the primary result and run the declared weight change; (b) inspect controls, edit the bio and keep both versions as exploration; (c) delete the failed result and publish the new score as the original; (d) raise feasibility to 4 because rank 2 looks disappointing.

Debrief. (a) is prespecified sensitivity; (b) is labelled development; (c) misrepresents the evaluation history; (d) abandons the evidence anchors. Any real predictive claim would need appropriate independent evidence. Do not invent new people or observations to manufacture that evidence here.

Run and challenge the workbook#

Save these sixteen files in one folder. The code uses Node’s built-in modules and needs no network access or package installation once downloaded.

• New: Week 11 cases, audit and evaluation functions, executable workbook.
• Candidate and invitation: Week 4 bios, Week 5 cases, Week 5 feasibility functions.
• Earlier protocol: Week 6 routing, Week 7 functions, Week 8 cases, Week 8 review functions.
• Offline and meeting records: Week 9 cases, Week 9 functions, Week 10 cases, Week 10 functions.
• Benchmark: 99 controls, generated reference scorer.

node week-11-worked-examples.mjs
node week-11-worked-examples.mjs --release-dir ./example-rc1

The first command writes a JSON report to the terminal. The second also creates a new directory containing index.html, evaluation.json, manifest.json and a short README. It refuses to overwrite an existing release directory. This is a teaching example, not a generated student submission. Save your own evaluated candidate and evidence in your release.

The report preserves the frozen input, per-control primary and weighted scores, rank decomposition, changed-case failure, time accounting, observation cutoffs, earlier cost agreement and exact/seeded Markov comparison. Its input manifest hashes all sixteen files. A candidate digest detects an accidental change against the retained record; it proves neither author identity nor consent. It also cannot prevent someone from replacing both the file and its digest.

Challenge — 5 minutes. In a disposable copy, change the conservative comparison to exclude ties, or include events exactly at the cutoff. Predict the literal answer that should fail before running the checks. Restore the source and reproduce the original report. Editing a candidate after freezing must create a new version, not a new digest masquerading as the old evaluation.

Prepare the release Week 12 can maintain#

Open the generated index.html. Its candidate paragraph comes from the verified frozen text with HTML escaping; the evaluation link opens the saved report. It identifies Alex as fictional and retains the primary rank 2/100 result. The changed-availability failure remains in the report. There is no real contact collection or app integration.

W3C’s page-structure tutorial supports logical headings and meaningful page regions. W3C’s Easy Checks provides an initial review of titles, text alternatives, keyboard focus and other accessibility concerns; it is not a complete conformance test.

Release exercise — 10 minutes plus discussion. Read the rendered candidate against the frozen source. Tab and Shift+Tab through the evaluation link; confirm visible focus and activate it. Inspect at 1920×1080 and 390×844, then resize and zoom. Check headings, legibility and horizontal overflow. Record actual browser/version, viewport, action, result and file version. A hash comparison does not prove that CSS displays the text, so retain both byte checks and rendered evidence. An automated accessibility pass likewise does not replace this review.

Re-run into ./example-rc2 and compare the two output files and manifests. Identical input bytes should yield identical output bytes; record the runtime version separately from this deterministic report. Keep source files, commands and dependency versions with the build. Week 12 consumes this exact release candidate for clean reproduction, a maintenance/exit manual and a demonstrated software rollback. Restoring an earlier HTML file cannot restore somebody’s agreement.

Save one Follow-up evaluation: labels and cutoff, 115-minute critique, primary failed target, common-weight sensitivity, separate changed-case failure, and the release with its actual browser evidence. This supports the existing final project, 50%, due 28 May 2027 at 5 pm; it adds no new essay or assessment. Keep one disappointing result and explain it. The dashboard is complete when the uncertainty is legible.

Continue to the Week 11 tutorial.

What Alex discovers#

Alex’s reference release keeps rank 2/100, its evidence and the separate stress-test failure. The cutoff label and the completed event record also keep their own meanings. The final problem arrives with the handover: someone else must rebuild exactly this release and recover it if a software update fails. Restoring a file must not restore an ended agreement.

Hand over Alex’s release in Week 12.