A client calls a downstream service and gets a connection timeout. It retries three times with exponential backoff and jitter before giving up. Why is the jitter necessary in addition to the exponential growth, and what can go wrong if retries are added without a cap on total attempts or a retry budget?

## How the delay is computed **Exponential backoff with jitter** governs how a client retries a failed call to a dependency. The mechanism starts with a base delay (say 100ms) and a multiplier (commonly 2x) applied per attempt, so successive retries wait roughly 100ms, 200ms, 400ms, 800ms, and so on, usually capped at a maximum delay so waits don't grow unboundedly, and capped at a maximum number of attempts so the client eventually gives up. The growth itself exists because a fixed short retry interval keeps hammering a dependency at a rate that doesn't decrease even as evidence accumulates that it's struggling; growing the delay backs off pressure the longer the trouble persists, giving the dependency more breathing room with each successive failure. ## Two ways to add the randomness Jitter adds randomness on top of that exponential value: - **"full jitter"** picks a uniformly random delay between zero and the computed exponential cap for each attempt; - **"decorrelated jitter"** (an approach AWS documented from production experience) computes each delay as a random multiple of the previous delay, bounded by the cap, which in practice spreads retries out somewhat more evenly than full jitter's occasional clustering near zero. ## Why jitter is separate from the growth The purpose of jitter is distinct from the exponential growth itself: many independent clients that all started failing at approximately the same moment (because they share the same downstream dependency that just degraded) would, without jitter, back off in lockstep - all waiting exactly 100ms, then all retrying simultaneously, then all waiting exactly 200ms, then retrying simultaneously again - producing synchronized traffic spikes timed exactly at each backoff interval. Randomizing each client's actual delay independently desynchronizes them so their retries smear out over time instead of arriving as a wave. ## The trade-off: resilience against load amplification The trade-off retries impose is between resilience and load amplification. A little bit of retrying absorbs the transient blips that are extremely common in distributed systems - momentary packet loss, a brief GC pause, a load balancer mid-rebalance - turning what would be a visible user-facing error into an invisible, slightly slower but successful response. But every retry is also additional load on the dependency, and that cost compounds badly across a multi-hop call chain: 1. if service A calls B, and B calls C, and each layer independently retries up to 3 times on failure, a single failure deep in C can, in the worst case, cause B to retry 3 times; 2. and each of those B attempts can itself trigger C to be called up to 3 times - the retry counts effectively multiply across hops rather than add. This is called a **retry storm**: exactly when a dependency most needs load to decrease, uncoordinated retries at every layer increase it, which can turn a partial degradation into a full outage, and can also prevent the dependency from ever recovering because the retry-amplified load never actually drops. ## The retry budget The standard mitigation beyond backoff and jitter is a **retry budget**: rather than only capping how many times one request retries, a retry budget caps the total proportion of retry traffic a client (or a whole service) is allowed to generate relative to its original request volume, tracked over a rolling window - for example, "retries may never exceed 10% of original request volume in any 60-second window." Once the budget is exhausted, further failures simply fail fast without retrying, which caps the maximum amplification factor system-wide regardless of how deep the call chain is or how many individual services are each retrying independently. ## Retrying what is not safe to retry A second major failure mode is retrying operations that aren't safe to retry. A network timeout on a request is ambiguous: - the server may have received and fully processed the request before the response was lost, - or it may never have received it at all. So blindly retrying a non-idempotent write (like "create an order" or "charge a card") risks duplicating the effect: two orders, two charges. The standard fix is an **idempotency key**: the client generates a unique key per logical operation and sends it with the request; the server persists which keys it has already processed and returns the original result for a duplicate key instead of re-executing the operation. Retries should also generally be restricted to error classes that plausibly indicate a transient condition - timeouts, connection resets, 503/429 responses - and never applied automatically to 4xx client errors, which indicate the request itself was invalid and will fail identically on every retry. ## Where the pattern is written down This is a well-established pattern documented in AWS's writing on exponential backoff and jitter, and implemented in most modern HTTP client and RPC libraries (e.g., gRPC's retry policies, resilience4j's Retry module) alongside **circuit breakers**, which are the natural complementary control: retries handle isolated transient failures, while a circuit breaker stops retrying altogether once failures are sustained rather than transient.