An operation like 'increment a user's loyalty-points balance by 10' or 'send a welcome email' is not naturally idempotent - running it twice produces a different (or duplicated) outcome than running it once. Given that the underlying operation can be delivered more than once, what general design techniques make it safe to apply repeatedly?

## Why some operations resist repetition Not every operation behaves like an idempotent `PUT` out of the box. - Some are **relative rather than absolute**: increment, append, decrement. - Some have side effects entirely outside the system's control (sending an email, firing a push notification), where 'running it twice' doesn't just risk a wrong number in a database - it risks an observable, irreversible duplicate action the user actually sees. Making these safe under at-least-once delivery requires deliberate design, not just wrapping them in a generic retry loop. ## The preferred technique: change the shape of the operation The first and generally preferred technique is to make the operation naturally idempotent by changing its shape from relative to absolute, and giving each intended effect a stable identity. Take 'add 10 loyalty points': instead of executing an unconditional increment on every delivery (which duplicates on redelivery, since running it N times adds 10N points), the system instead records a transaction with a stable ID derived from the triggering event: - an order ID - a message offset - anything unique to 'this specific reason for +10 points.' If that record is re-attempted with the same ID, a unique constraint on the transaction ID makes the second write a no-op, and the user's actual balance is computed as a derived sum, or maintained incrementally but only advanced once per unique transaction ID via an upsert. This pattern - record discrete, uniquely-identified facts rather than mutate a running total directly - is the same idea behind **event sourcing** and behind idempotent financial ledgers: you never mutate the balance in place from an unguarded delta, you append an identified entry and derive or upsert the total. ## The 'set' variant: conditional and versioned writes A closely related technique for operations that are naturally 'set' rather than 'add' is to use **conditional/versioned writes**: an update guarded by an expected-version check (optimistic concurrency control), or a write that's naturally absolute ('set the resource's shipping status to shipped' rather than 'advance the shipping status by one step'). Re-applying an absolute set is safe by construction, because applying it twice leaves the same end state as applying it once - the literal definition of idempotence. ## The fallback technique: gate the effect behind a dedup record The second technique, needed when the side effect genuinely cannot be made naturally idempotent - sending an email is the canonical example, since there is no way to upsert an email that's already left an SMTP relay - is to gate the operation behind an explicit **dedup record**, structurally identical to the idempotency-key pattern used for HTTP APIs. Before sending, the system checks (or atomically claims, using the same claim-pattern used for concurrent duplicate requests) a record keyed by something like `(user_id, event_type, event_id)` - 'welcome email for user U, triggered by signup event S.' - If that record already exists, skip the send. - If not, atomically claim it, send the email, and mark it sent. This doesn't make sending inherently repeatable; it prevents the system from attempting to repeat it in the first place, by remembering that the intended effect already happened. ## Which one to reach for The trade-off between these two techniques is where correctness lives. | Technique | Behavior when the bookkeeping is lost | | --- | --- | | **Natural idempotency** - the transaction-ledger / absolute-write approach | Generally more robust, because it tolerates gaps in the dedup bookkeeping: even if a 'processed' marker is somehow lost, re-deriving state from uniquely-identified facts self-heals, since replaying the same fact twice is harmless by construction. | | **Dedup-gated operations** - the email-send approach | Strictly dependent on the gate itself being correct and durable; if the 'already sent' record is lost, corrupted, or has too short a TTL, the side effect will genuinely repeat, with no self-healing possible after the fact - you cannot un-send an email. | This is why, wherever the operation allows it, engineers prefer to redesign toward natural idempotency rather than relying purely on a dedup gate, and reserve the dedup-gate approach for operations where no natural-idempotency redesign is possible. ## The same advice from a platform A concrete real-world instance: **AWS's** own guidance for Lambda functions triggered by at-least-once event sources explicitly recommends this two-pronged approach - - use idempotent operations (conditional writes, upserts) wherever the downstream effect allows it; - fall back to storing processed-event IDs with a TTL as an explicit dedup gate for effects, like third-party API calls or notifications, that can't be made naturally idempotent.