What is covered, week by week, and which topics each lecture draws on.
Lectures#
Twelve lectures connect mechanisms, public evidence, and the decisions you must make at 32M scale — from what a language model learns, through pre-training, post-training, and specialisation, to packaging a bounded claim. Each lecture page names the teaching team member presenting it and links to the weekly session and project it feeds.
What a Language Model Learns
Probability over text, next-token prediction, maximum likelihood and cross-entropy, n-gram intuition, and perplexity as a first, limited piece of evidence
Week 1 · 22 February 2027
Inside a Decoder-Only Transformer
Encoder, encoder-decoder, and causal decoder-only attention; tracing tensor shapes through a decoder block; architecture as a 32M parameter allocation
Week 2 · 1 March 2027
Scale, Data, and Optimisation Under Fixed Compute
Power-law intuition, Kaplan- and Chinchilla-style compute allocation, data quality versus data quantity, and reading a training pilot honestly
Week 3 · 8 March 2027
Evaluate, Decode, and Package a Base Model
Hold-out and contamination, checkpoint selection, decoding as an experimental variable, and reproducible submission packaging
Week 4 · 15 March 2027
From a Base Model to a Target Behaviour
Capability versus elicited behaviour, continued pre-training versus SFT versus preference learning, and a measurable narrative-voice target
Week 5 · 22 March 2027
Supervised Fine-Tuning and Data That Teaches Behaviour
Response-only loss masking, instruction schemas, synthetic demonstration data, and learning without catastrophic overwriting
Week 6 · 29 March 2027
Learning from Preferences Without Hiding the Cost
Bradley–Terry preference data, reward models and RLHF, RLAIF and DPO, and proportionate preference learning at 32M parameters
Week 7 · 12 April 2027
Did the Behaviour Change, and What Regressed?
Multi-objective post-training evaluation, blind A/B review, paraphrase robustness, and checkpoint selection against a pre-declared priority
Week 8 · 19 April 2027
Reasoning as Generated and Verifiable Behaviour
Chain-of-thought as generated intermediate tokens, self-consistency, exact-answer verification, and rationale faithfulness
Week 9 · 26 April 2027
Train a Specialist That Can Be Tested
Task contracts, frozen starting checkpoints, split integrity, and full versus parameter-efficient adaptation for Project 3
Week 10 · 3 May 2027
Generalisation, Supervision, and One Honest Ablation
Template and paraphrase shift, outcome versus process supervision, verifiable rewards as frontier context, and a matched ablation design
Week 11 · 10 May 2027
Audit the Training System
Reconstructing the artifact chain, reproducing the core metric, what transfers across scale, and the final engineering retrospective
Week 12 · 17 May 2027