A client retries an append call to an event store after a network timeout, but the original append had actually already succeeded server-side before the timeout. How do idempotency keys prevent this from creating a duplicate event?

## Why a timeout is ambiguous The core problem idempotent append solves is that a network timeout is fundamentally ambiguous from the client's point of view. When a client calls `append(streamId, expectedVersion, events)` and the connection times out before a response arrives, the client cannot distinguish between three very different outcomes: - **(a)** the request never reached the server, - **(b)** the request reached the server and was rejected, or - **(c)** the request reached the server, was committed successfully, and only the response was lost on the way back. In cases (a) and (b) it's safe (even necessary) to retry the exact same append. In case (c), retrying naively means appending the same logical event a second time — and because an event store's default behavior on a fresh append call is simply to append, without any built-in notion of 'have I seen this exact write before,' a naive retry produces a genuine duplicate: two `MoneyDeposited(amount: 100)` events in the stream where there should be one, silently inflating the derived balance by 100 the next time the stream is replayed. ## How idempotent append closes it Idempotent append closes this ambiguity by attaching a stable identity to the write itself, independent of the network transport. 1. The most common mechanism is to have the client generate the event's ID (often a UUID) **once, before the first attempt**, rather than letting the server assign it — so a retry of the same logical operation carries the identical event ID as the original attempt, not a new one. 2. The event store persists this ID alongside the event when it commits the append, typically indexed for fast lookup. 3. On any append call, the store checks whether it has already committed an event with that exact ID in that stream; if so, it treats the call as a **no-op duplicate** and returns the same success response the original commit would have produced (often including the position/version the event actually landed at), rather than inserting a second event or throwing an error. From the client's perspective, retrying a timed-out append is now always safe: either the retry is genuinely the first successful attempt (server never got it), or it's a harmless duplicate detection that confirms 'yes, that already happened, here's where.' ## Why a duplicate event is worse than a duplicate row This matters more in event sourcing than in a typical CRUD system because the event log is the literal source of truth that all derived state is computed from — a duplicate row in a mutable table is often self-correcting (the next `UPDATE` just overwrites it again), but a duplicate event is **not self-correcting**: it's a permanent, immutable extra fact that every future replay will fold in, permanently double-counting the deposit, permanently double-shipping the order, forever, unless someone notices and appends a manual correction. The blast radius of a duplicate event is also wider than a single table row: every projection/read-model built by consuming that stream inherits the duplication independently, so fixing it after the fact means correcting the event store and every downstream materialized view. ## The trade-off The trade-off idempotent append introduces is bookkeeping cost and a design constraint: - the store (or a companion table) needs to **track seen-IDs per stream** indefinitely or for some bounded retention window, - and clients must be disciplined about **generating the ID once and reusing it verbatim across retries** rather than accidentally minting a fresh ID on each attempt — which is a surprisingly easy bug to introduce if the event ID is generated inline at call time instead of being fixed earlier in the request's lifecycle (e.g., at the point the command was received, before any retryable I/O). There's also a scope question: idempotency keyed per-stream is enough to prevent double-appends to that stream, but if the same logical business operation could legitimately touch multiple streams (a transfer debiting one account and crediting another), true end-to-end idempotency needs a higher-level mechanism — often an outbox/saga pattern with its own dedup key — spanning both appends. ## A concrete failure mode A concrete failure mode: a payment service calls append with a freshly-generated UUID for the event ID on every call, including retries, because the UUID generation sits inside the retry loop instead of outside it. Every network hiccup then produces a genuinely distinct event ID on retry, so the store's duplicate-detection has nothing to match against, and the 'idempotent' append silently isn't — the team only discovers this when reconciliation between ledger totals and bank statements starts drifting under load. ## Where it shows up A concrete real-world example: EventStoreDB deduplicates appends within a stream based on the event's ID field specifically for this reason — client SDKs are documented to generate the event ID client-side, once, precisely so that transport-level retries (which the client library issues automatically on transient network errors) land on the same ID as the original attempt and get deduplicated server-side rather than double-appended.