Two retries carrying the same Idempotency-Key HTTP header arrive at your API at the same moment, before the first has finished. What must the server guarantee, and how do you implement it?

## The scenario A client times out and retries. Or a mobile app fires two requests from a double tap. Or a load balancer duplicates a request across two backends. Two requests with the same `Idempotency-Key` are now being handled simultaneously by different processes, and the first has not written a result yet. If both execute, the customer is charged twice — the exact outcome the pattern promised to prevent. This is where naive implementations fail. Storing keys is easy; handling the concurrent window is the real engineering. ## Why check-then-act loses ``` row = SELECT * FROM idempotency WHERE scope=? AND key=? if row is null: result = chargeCard() INSERT INTO idempotency (...) ``` Both requests run the SELECT before either runs the INSERT. Both see null. Both charge. The window is small, which is exactly why this bug survives testing and surfaces in production during a network blip that makes clients retry en masse. ## Claim by writing Make the **insert** the claim: ``` INSERT INTO idempotency (scope, key, fingerprint, state, lease_expires_at) VALUES (?, ?, ?, 'in_progress', now() + interval '60 seconds') -- unique index on (scope, key) ``` The database's unique index is the arbiter. One request gets a row; the other gets a unique-constraint violation, which is not an error condition but information: *someone else owns this key*. No distributed lock service is required — the constraint you already have is the mutual exclusion primitive. Only the winner proceeds to execute. ## What the loser returns Three defensible designs: **409 Conflict** with a specific error code such as `idempotency_key_in_progress`. This is what Stripe does. It is honest, cheap, and holds no server resources. The client is already a retrying client, so it backs off and tries again, and by then the record is completed and it gets a replay of the real response. Downside: the client sees an "error" for what is actually a success in flight, so the error must be documented as retryable. **425 Too Early.** Semantically closer to "come back later" and less likely to be logged as a failure by naive clients, but far less common, so clients may not handle it specially. **Block and then replay.** The loser waits (polling the row, or on a condition variable / advisory lock) up to a short deadline, then returns the completed stored response. The client experience is best — it gets the real answer on the first call. The costs are real though: you hold a request thread or connection for the duration, which is a availability hazard under a retry storm, and you need a hard deadline after which you fall back to 409 anyway. Reasonable when the operation is fast and the client fleet is large and unsophisticated. What is **not** acceptable: executing the operation, or returning a synthetic success. ## Crashed in-progress rows If the winner's process dies after inserting `in_progress` and before writing the result, the key is now claimed by nobody. Without recovery, every future retry sees `in_progress` and gets 409 forever — a permanently poisoned key, and an operation the client can never complete or confirm. The fix is a **lease**: store `lease_expires_at` on the in-progress row. A later request that finds an expired lease may take over the key (atomically, with a conditional update that re-stamps the lease so only one taker wins). Set the lease longer than the maximum plausible execution time, because taking over a key whose original execution is still running reintroduces double execution. The deeper safety net for money operations is **reconciliation with the downstream system**: pass your key through to the payment provider as their idempotency key, so a takeover that re-calls them cannot double-charge — the provider deduplicates on the same key. ## Atomicity of completion Write the business effect and the transition to `completed` in one transaction whenever they share a database. Otherwise: - Mark completed first, then the effect rolls back → replays report a success that never happened. - Effect commits, then completion write fails → a retry re-executes, double effect. Co-locating the dedupe table with the business data buys you this for free and is the main argument against putting the dedupe store in a separate cache. ## What to say in an interview "At most one execution per key. I claim the key with an INSERT under a unique index before any work, so the database arbitrates instead of an application-level check-then-act. The loser returns 409 with a retryable in-progress error rather than executing. In-progress rows carry a lease so a crashed request doesn't poison the key, and the effect and the completed record commit in the same transaction so a replay never reports work that rolled back."