Two identical retry requests carrying the same client-generated idempotency key arrive at two different, stateless replicas of an API server within a few milliseconds of each other - before either has finished writing to the shared idempotency store. What race condition can occur, and what pattern prevents it?

## The naive sequence This is the classic **check-then-act race condition**, applied specifically to idempotency/dedup stores. The naive implementation of 'process once per key' looks like: 1. read the store for the key; 2. if absent, execute the operation; 3. write the key and result to the store. Under concurrency, this sequence is not atomic as a whole, even if each individual read or write is. If two requests carrying the same idempotency key arrive close enough together - say, hitting two different stateless server replicas behind a load balancer, both retries fired by an aggressive client-side retry policy - both can execute their 'read' step before either has performed its 'write' step. Both reads come back empty, both requests conclude 'nobody has handled this key yet,' and both proceed to execute the real side effect. The dedup mechanism, which exists specifically to prevent this, fails precisely because its own bookkeeping wasn't protected against concurrent access. ## Collapsing check and claim into one atomic step The fix is to collapse 'check' and 'claim' into a single atomic operation, so that only one of the concurrent requests can ever win the right to execute the side effect, and every other concurrent request is guaranteed to observe that the key is already claimed. The most common concrete mechanisms: - a database `INSERT` with a unique constraint (or primary key) on the idempotency-key column - both requests attempt to insert a row for the same key; the database's constraint enforcement guarantees exactly one of the two INSERTs succeeds and the other fails with a uniqueness-violation error, atomically, regardless of timing; - in a cache-backed store, the equivalent is a single atomic **set-if-not-exists** command. The winner of this race is now the sole owner of executing the business logic; the loser must not proceed to do the work itself. ## What the loser does next What the loser does next is a real design decision with several valid answers depending on latency requirements. 1. One option is **synchronous polling**: the loser waits (with backoff) for the winner to finish and write the final result into the same row, then reads and returns that result - this gives the caller a normal-looking response but adds latency and requires the winner's row to transition through a 'processing' state to a 'completed' state that the loser can poll for. 2. A second option is to immediately reject the loser with a **409 Conflict** or similar response telling the client 'this request is already in flight, retry after a short delay' - simpler to implement, pushes the wait back to the client's own retry logic. 3. A third, used by some payment APIs, is to have the loser's request block on a short-lived **distributed lock** tied to the key rather than the dedup row itself, converging to the same effective behavior as polling. ## The stuck-'processing' hazard There's a secondary correctness detail worth calling out: the 'processing' state itself is a hazard. If the winning request crashes after claiming the key (the INSERT succeeded) but before finishing the side effect and writing the final result, the row is stuck in 'processing' forever, and every future retry will see 'claimed, but no result yet' and either wait indefinitely or be rejected forever, effectively wedging that operation. Production-grade implementations handle this with a **claim timeout**: if a row has been in 'processing' for longer than some bound, a subsequent request is allowed to re-claim it and retry the operation, on the assumption that the original claimant crashed rather than being merely slow. This reintroduces a small window for a duplicate if the 'crashed' claimant was actually just very slow and completes after the reclaim - a trade-off between availability (not wedging forever) and strict duplicate prevention that has to be tuned to the specific operation's latency profile. ## The documented reference **Stripe's** public API is a frequently cited real-world reference for this exact pattern: their documented idempotency behavior returns a 409 Conflict if a second request with the same key arrives while the first is still being processed, rather than letting the second request race ahead - the 'reject the loser' strategy described above, paired with a bounded processing-lock lifetime so a crashed request doesn't permanently block retries of that key.