Multiple independent clients all start retrying a failing call to the same dependency using the exact same exponential backoff schedule. Why can this still overload the dependency, and how does adding 'jitter' — specifically full jitter versus equal jitter — help?

## Backoff alone leaves clients correlated Exponential backoff solves the 'how fast should I retry' problem for a single client, but it does not by itself solve a second, distinct problem: correlated retries across a large population of independent clients. Picture a dependency that has a brief availability blip lasting one second, during which every one of a thousand concurrent callers gets a failure at roughly the same moment. If all thousand clients use the identical deterministic exponential schedule — say 1s, 2s, 4s, 8s — then every single one will retry at t+1s, and if that retry also fails (because the dependency, now facing a synchronized spike of a thousand simultaneous requests, buckles under it), all thousand will retry again at t+3s (1+2), then t+7s, and so on. The backoff schedule successfully spaces out each individual client's own retries, but it does nothing to desynchronize different clients from each other — they all 'wake up' together, recreating exactly the **thundering-herd** problem backoff was meant to solve, just delayed and repeated at each retry round instead of happening once. This failure mode is easy to miss in testing (where a single client is exercised in isolation) and only shows up under real production load with many concurrent callers. ## The two most commonly discussed strategies Jitter fixes this by injecting randomness into the delay so clients that failed at the same instant do not retry at the same instant. There are several standard jitter strategies, and the two most commonly discussed are full jitter and equal jitter, both described in AWS's widely-cited 'Exponential Backoff and Jitter' architecture blog post. - **Full jitter** computes the exponential cap for the current attempt (`cap = min(maxDelay, base * factor^attempt)`) and then picks the actual delay as a uniform random value between 0 and that cap: `delay = random(0, cap)`. This maximizes the spread of retry times across the population, since some clients will retry almost immediately and others will wait nearly the full cap, which empirically produces the lowest total number of retries and the shortest aggregate completion time across a population of failing clients, per AWS's own simulation results. Its downside is that any individual retry might land very close to zero delay, so a single client can occasionally retry almost immediately after a failure. - **Equal jitter** addresses that specific concern by keeping half of the computed exponential delay as a fixed floor and randomizing only the remaining half: `delay = cap/2 + random(0, cap/2)`. This guarantees every client waits at least some minimum amount before retrying, trading away some of full jitter's spread (and therefore some throughput advantage) for a more predictable lower bound on retry timing. ## Spread versus predictability The trade-off across jitter strategies is spread versus predictability. | Randomization | What it gives you | |---|---| | More randomness (full jitter) | minimizes the odds of synchronized retry waves and generally produces the best system-wide outcome, but makes any single client's retry timing highly variable, complicating reasoning about worst-case latency for a specific request | | Less randomness (equal jitter, or no jitter) | is easier to reason about per-request but reintroduces correlation risk across the client population | The choice usually favors more jitter for backend-to-backend service calls at scale — where system-wide load pattern matters more than any single request's exact timing — and can favor equal jitter for latency-sensitive paths with fewer concurrent callers, where the goal is a predictable floor on retry delay. ## The signature on a dashboard Failure modes without jitter show up as periodic load spikes on a dependency's dashboard at intervals matching the backoff schedule (a spike every ~1s, ~3s, ~7s after an initial blip, tapering as clients give up or succeed) rather than a smooth decay — a recognizable signature distinguishing 'backoff without jitter' incidents from plain retry-storm-without-backoff incidents, which instead show sustained elevated load with no periodicity. ## Where it shows up A concrete real-world example is AWS's own guidance for its SDKs: default retry behavior in recent AWS SDK major versions uses exponential backoff with jitter specifically because AWS observed synchronized retry spikes from large customer fleets during transient service degradations, and jitter was the fix that spread that load out smoothly instead of in discrete waves.