Week 1: Welcome to calibration

What a calibration meeting is, who is in it, what the curve looks like, and why a company that knows a rating is mostly the rater still runs one.

23 February 2027

Open the slides

Why it worksOver half the variance in a rating is the rater, not the rated; calibration exists to squeeze that out, and it does so by giving your manager three minutes.

Under Jack Welch, General Electric sorted every employee into a 20/70/10 curve each year and let the bottom ten percent go. Microsoft ran its own version, made explicit in a 2011 memo with five buckets and a fixed percentage for each; a 2012 Vanity Fair piece found that every current and former employee it interviewed called it the most destructive process in the company, and in November 2013 Microsoft dropped it. In January 2025 Meta cut about five percent of its staff as “low performers”, and in May told managers of large teams to rate fifteen to twenty percent of them below expectations, up from twelve to fifteen the year before.

Here is the fact the room is built on, and the one this course keeps coming back to. In 2000 Scullen, Mount and Goff took two samples of more than two thousand managers, each rated by several bosses, peers and direct reports, and asked how much of a rating was the person being rated. The answer was less than half. Sixty-two percent of the variance in one sample and fifty-three in the other came from the rater: who they were, what they happened to value, how generous they were that year. Deloitte found the same thing in its own data in 2015, counted two million hours a year spent on reviews, and rebuilt the whole system around the finding.

Calibration is the fix. If one manager’s rating is mostly that manager, put eight managers in a room and make them argue, and the idiosyncrasies should cancel. The one field study of calibration meetings, in 2019, found that the managers who sit in them believe it works: a shared frame of reference, more accountability, more information per name. Nothing in this course says they are wrong. What the course says is what the fix costs. Eight managers, forty names, a fixed distribution and one afternoon means three minutes a name, and three minutes rewards whoever hands their manager the best three-minute case. Every trick in this course is a way of doing that. The room was built to take the rater out of the rating, and it runs on your manager’s advocacy, which is the rater with a time limit.

You are represented by your manager’s account of you, for about the time it takes to read this page.

Outline

Sources

Teaching team