Finding the point of no return#
Week 11 — SLOP3092
← Back to the lecture pageLast week#
Attempts per request — the number that moves while the success rate stays flat.
This week: a sequence that got halfway#
Five steps, in this order:
- Reserve the stock
- Charge the card
- Send the confirmation email
- Release the reservation lock
- Write the order row
Step five fails.
The reflex#
Replay the whole operation from the top.
Eleven weeks made exactly that safe. This call has an idempotency key, a schedule with jitter, a shared budget and a classification table.
The email has already been sent#
The two steps that already happened#
Week 2’s key covers the charge, so the card is not charged twice.
It does not cover the email. That has left your system, so the second run sends a second confirmation to a customer whose order still does not exist.
The lock is already released, so the stock may now be reserved by somebody else — and the replay can fail on a step that succeeded the first time.
The fix#
Find the step with an effect outside your system that you cannot recall, and move it after everything that can still fail.
Reserve, charge, write the row, then email. Release the lock in a cleanup path that runs either way.
A retry then either never reaches the irreversible step, or reaches it exactly once.
When the ordering is not yours#
A payment provider that emails the customer itself. A partner API that despatches on receipt.
What is left is a compensating action, which is a different thing: a refund is not an un-charge and a correction email is not an un-sent email. The customer sees both.
Who decides#
nobody.
By the time any of client, library, platform, operator or product could decide, the effect has already left the building. There is no retry policy that undoes it.
The only move available is upstream, in the ordering — which is why this is the one week with no owner to name.
In the lab#
- Mark each step of a five-step operation reversible or not
- Find the point of no return
- Reorder so it comes last, inject a failure after it, and confirm the retry is safe
- Then make one irreversible step belong to a third party and write the compensating action
Next week#
Twelve weeks of mechanisms.
Next: whether the retry should have happened at all.
← Back to the lecture page