Eleven weeks#

Idempotency keys. Backoff. Jitter. Shared budgets. Breakers. Deadlines.

Each one has a limit built into how it works, and each one is the right answer somewhere.

This week#

A failure from earlier in the semester, handed back with the outcome removed.

Something failed. Somebody retried. The account stops before saying whether that helped.

The reflex#

Assume the retry was the villain.

Eleven weeks of this course have mostly been about retries going wrong.

Deciding before reading#

Week 1’s retry was correct. So was week 2’s, once the key was in place.

The reflex to distrust retrying can be just as unexamined as the reflex to trust it was, in week 1.

Reading it cold#

Say what the retry actually did: how many attempts reached the dependency, what they added to the latency of calls that were already working, what state a failed attempt left behind, who could see it happening.

Then say what you would have decided knowing only what was knowable at the time.

Hedging is the easiest answer to write#

It is also the one that scores lowest here.

You commit to a position you can be wrong about, in front of somebody assigned to argue the other one.

Who decides#

product.

Engineering can measure both sides of this and still not settle it. Whether a checkout flow should add 400 ms to its 99th percentile so that one order in a thousand stops failing depends on what the product is for. That has been true since week 1; it only ever looked like a question about mechanism.

In the lab#

1. Take the case
2. Argue for or against having retried
3. Name what you would measure to know if you were right
4. Defend the argument against a partner assigned to take the opposite side