Week 8 Tutorial: Threat Modeling and Anomaly Detection

Detect bots, financial scams and identity inconsistencies in synthetic cases. Calculate the cost of false alarms before trusting a flag.

29 April 2027

Inputs#

Bring the Communication state machine and the boundary map from your week 2 Platform audit. Retain the candidate and invitation versions, G1’s supplied quote, event IDs, two-hour observation window and closure reason. Work with invented case cards, never real identities. Solo implementation and review are accepted.

Open the lecture’s six-file offline workbook. Its 100 scored rows and seven triage cards are separate authored fixtures. Scores are arbitrary points, not probabilities; M/B row IDs reveal answer labels and must not become predictor features. No reverse-image search or third-party image upload is part of this exercise.

Finish with The flag factory requests a verdict button: the interactive quiz. Six new cases include coded observations, detector tables and a population diagram. Apply the week’s evidence and error-cost rules, then submit all six responses to unlock the score and worked explanations. The quiz is practice and carries no course marks.

Follow the 60-minute lecture selection and independent-work plan. Complete the preparation and open the workbook before Thursday. The four blocks below are the entire 90-minute tutorial; use existing lecture or study working for review and refinement. Remaining pack material belongs to the week’s independent study allocation.

90-minute tutorial#

1. 20 minutes — map threats. Spend five minutes recovering the Week 2 boundaries, ten drawing the path from a fictional message to a proposed address/payment form, and five reviewing the controls. Name assets, crossings and actor opportunities for automation, a financial scam and identity misrepresentation. Keep observations separate from accusations. Output one labelled flow and a control for each asset.
2. 25 minutes — calculate the detector. Spend eight minutes filling the original matrix and denominators, eight recounting thresholds 50 and 75, and nine comparing losses and assumptions. Verify TP = 8, FN = 2, FP = 18, TN = 72 at score ≥ 50. Calculate precision 8/26, recall 8/10 and false-positive rate 18/90. Compare L = 5FN + FP, then change the missed-threat cost to 10. Record which tested threshold wins and why the answer changes.
3. 20 minutes — challenge heuristics. Spend eight minutes classifying the seven authored cards, seven writing evidence/uncertainty/action records for two contrasting cases, and five reviewing the weakest inference. Compare typo-only with transfer plus identity conflict. Explain why an image match, a non-match, syntax and a timeout remain uncertain. Run screenCase on supplied codes; do not present it as a free-text classifier or treat disclosed automation as malice.
4. 25 minutes — protocol review and quiz. Use fifteen minutes to review your prepared implementation: six replaying the three review histories, six checking literal assertions and three exchanging a critique or writing a solo review. Put review around the existing Week 7 snapshot. Require all finding IDs and a note to clear a hold; retain incoming replies and observation expiry; preserve closed as terminal. Run the contact/payment release checks. Record a failing assertion from one deliberate mutation, restore it and verify the restored result. Use the final ten minutes for the six-case quiz: about eight minutes to respond and two to review the explanations after all cases are answered.

Worked checkpoints#

The cost formula omits ordinary review overhead, so retain flag counts beside loss. Comparing two thresholds does not find a global optimum. If time remains, compute the 50/75 tie at cFN = 6.5 or the lecture’s 1%-prevalence projection: TP 8, FN 2, FP 198, TN 792 in 1,000 hypothetical cases.

Literal checks before polish#

Assert the four original cells and a score exactly equal to the cutoff. Assert that no flags produces undefined precision (null), rather than perfect precision. Reject missing numeric labels instead of treating them as benign.

For the event machine, assert the complete refused snapshot stays unchanged after clearance or a late reply. A review hold must reject a first send even if the underlying phase is ready. A no-match card and a read receipt must not remove an earlier finding. Incomplete finding IDs, empty review notes, conflicting IDs and old timestamps must fail.

Temporarily change ≥ to >, or remove the closed guard. Confirm the intended edit matched, observe the corresponding literal assertion fail, restore the code, and run again. Use the worked lecture debrief to explain what the failure establishes. These checks validate the fixture’s behaviour, not detector quality on real people.

Include a test rejecting a request to collect real contact details or payments:

checkReleasePlan({ ...releasePlan, collectsRealContact: true }).allowed === false
checkReleasePlan({ ...releasePlan, acceptsPayments: true }).allowed === false

Turn these expected expressions into assertions in your implementation. Also inspect the actual release: a correctly declared object cannot prove that a separate form or network call follows it. Keep third-party image uploads, app integration and published accusations out of the fictional release.

Deliverable: Threat model#

Save the asset/boundary diagram, original confusion matrix, at least two threshold choices with explicit error costs, two evidence/uncertainty/action records, and state-machine changes with the three literal traces. Include the contact/payment rejection tests and one documented mutation/restoration result. Carry the unchanged version references and closure reason. These artefacts form one Threat model, not a new essay or extra assessment.

Week 9 uses these boundaries to decide what information a proposed offline meeting actually needs. Hand over a proposal and its unresolved conditions; review clearance is neither acceptance nor a safety certificate. Alex’s Friday 5–7 pm, bus and $20 constraints remain fixed, and recipient availability remains unknown.

Interactive quiz · 6 cases · about 8 minutes + 2 to review

The flag factory requests a verdict button

Six review tickets. The dashboard is already writing its victory speech.

Make a decision in every case, then select Reveal answers. You can change any response before submitting. Scores and explanations stay hidden until the whole quiz is answered.

All requests, labels and observations are authored teaching fixtures. Cases 3–4 share a new 100-row detector sample; the cards, population projection and review log are separate exercises. Scores are arbitrary points, not probabilities. Practice only: no course marks. Responses stay on this page; reloading starts a fresh attempt.

The cases are readable without JavaScript. Enable JavaScript to navigate, submit your responses and unlock the explanations.

Case 01 / 06The reservation form collects souvenirs

Trace assets and test the actual release

A fictional reservation mock-up says “Upload a student-card photo and pay a $6 deposit to hold the game table.” Its release-plan object declares no image uploads and no payments, so checkReleasePlan returns allowed. Page review nevertheless finds working links to an external upload form and checkout. No upload or payment has been made.

Which repair addresses the actual boundary crossing?

AKeep the links because the release-plan object passed; the declared Booleans certify the page.BRemove the external upload/payment paths and use written fictional cards. Record which assets would leave the prototype and check the page again.CKeep the links for testing and publish the counterpart’s name as a confirmed scammer.DKeep only checkout: a $6 payment is small enough to fall outside the course’s release rule.

Case 02 / 06A typo joins the threat committee

Use supplied observation codes without inventing intent

A reviewer manually codes these three new cards. Feed the supplied codes and quotes to screenCase; it does not classify free text. None of the cards has an earlier finding or a refusal.

New authored cards · observation code followed by its evidence

Card	Supplied observations
A	disclosed-automation: “Automated rules helper.” syntax-only: “I enjyo co-op games.”
B	image-no-match: “No item for IMG-Q8 in the supplied fictional index.”
C	money-request: “Transfer $6 to reserve the game table.”

Which triage result follows the declared rule?

AReview all three: automation, a typo and an absent image result are each proof of deception.BReview C; certify A and B as safe because neither activates a review code.COnly C adds a review finding. A and B add no flag; that result certifies neither authenticity nor safety.DLabel C malicious and discard its evidence quote because a money request settles intent.

Case 03 / 06Six correct flags borrow the wrong denominator

Calculate precision from the flagged population

This new 100-row authored sample uses flag when score ≥ 55. The rows below are its label counts after prediction, separate from the lecture’s original 100 cases. A dashboard claims “75% of our flags are correct” by dividing six by eight.

Authored label versus prediction · counts, not detector scores

Label	Flag	No flag
Malicious	6	2
Benign	10	82

Model supplied
Precision = TP / (TP + FP). For a percentage, multiply by 100. The denominator includes every flagged row.

What percentage of flags have the authored malicious label? Enter the percentage as a number, without a % sign.

Precision (%)

Enter one number. A calculator and your lecture notes are welcome.

Case 04 / 06The quieter queue submits an expense claim

Compare thresholds under explicit error costs

Raise the threshold from 55 to 75 on the same fixed rows as Case 3. The error counts become those below. Only these two thresholds are being compared; the loss model omits ordinary review overhead.

Same authored sample · FN means missed malicious label; FP means benign flag

Threshold	FN	FP
55	2	10
75	4	2

Model supplied
L = cFN × FN + cFP × FP. Keep cFP = 1 error unit. Compare cFN = 4, then cFN = 6.

Which cost comparison should replace “the higher threshold is always better”?

A75 wins at both costs because it always produces fewer false alarms.B55 wins at both costs because missing a malicious label always dominates any number of false alarms.CBoth thresholds tie at both costs; changing an error cost cannot change the ranking.DAt cFN = 4 both cost 18. At cFN = 6, 55 costs 22 and 75 costs 26, so 55 wins among the two tested choices.

Case 05 / 06The detector moves to a quieter neighbourhood

Project precision under a changed base rate

In a separate hypothetical population of 1,000 cases, malicious-label prevalence is 2%. Assume recall stays at 60% and the false-positive rate stays at 5%. These conditional rates are supplied assumptions, not estimates from Cases 3–4.

Model supplied
True flags = malicious cases × recall. False flags = benign cases × false-positive rate. Precision uses both types of flag.

Which projected report uses the right denominator?

A12 true flags and 49 false flags: precision 12/61 ≈ 19.67%, under the fixed-rate assumptions.B12 true flags out of 20 malicious cases: precision is 60% in the new population.C49 false flags out of 980 benign cases: precision is 5% in the new population.D943 correct predictions out of 1,000 cases: precision is 94.3% in the new population.

Case 06 / 06The clearance stamp meets a closed file

Retain findings, observation history and terminal closure

Retain a Week 7 snapshot with send at 4 h, wait at 4.1 h and a two-hour window. Assess a money-request card at 4.2 (finding f-pay) and an image-match card at 4.3 (finding f-image). Both findings are open. Each row is a separate continuation from that held snapshot. Every resolution below names both finding IDs and supplies a non-empty authored review note.

Independent branches · new event IDs and chronological observation hours

Branch	Next events
A	Reply 6.2; resolve review 6.4
B	Decline 5; resolve review 6.4
C	Assess image-no-match card 6.1

Which summary preserves the review and communication contracts?

AA clears to replied/reply by restarting the observation window at clearance; B and C also resume normally.BA clears to replied/no-reply. B stays completely unchanged after closing on decline. C stays in review with underlying pending/no-reply.CA can clear using f-pay alone; B’s full clearance reopens it; C’s non-match erases f-image.DReview freezes all incoming events and observation time, so A, B and C remain pending/open until clearance.

Reveal answers

Answer all 6 cases to unlock your score and the worked explanations.