skip to content

Why does exponential backoff with time.sleep need random jitter and a delay ceiling?

level: middleimportance: should knowfreq 55%

answer

  1. The wait has to grow, not repeat
  2. Identical callers must not agree on timing
  3. Doubling needs an upper bound
  4. A uniform draw inside the current window
  5. min(cap, base * 2 ** n), then randomise

basics

~20 s

Exponential growth gives a struggling dependency progressively more room instead of hammering it. Jitter randomises each wait so callers that failed at the same moment do not retry in lockstep, and a ceiling stops the doubling from turning into multi-minute sleeps.

solid answer

~50 s

The delay for attempt *n* is `min(cap, base * 2 ** (n - 1))`: each failure doubles the wait, and the `min` clamps it so attempt eight does not sleep for four minutes. Then randomise it — full jitter is `random.uniform(0, window)`, which spreads retries across the whole window. Without jitter, every caller that failed during the same outage wakes at the same instants and lands on the recovering dependency as one synchronised wave, repeatedly. Pass the delay to `time.sleep` in synchronous code and `asyncio.sleep` inside a coroutine, since `time.sleep` blocks the whole event loop thread. Bound the retry by a total deadline measured with `time.monotonic`, not only by an attempt count: the sum of the delays plus the per-attempt timeouts is what the caller actually waits, and that number belongs in your latency budget.

code

python · 12 lines
python
import random


def delay_schedule(attempts=7, base=0.2, cap=5.0, rng=None):
    rng = rng or random.Random()
    for attempt in range(attempts):
        window = min(cap, base * 2 ** attempt)
        yield round(rng.uniform(0, window), 3)


print(list(delay_schedule(rng=random.Random(0))))
print(list(delay_schedule(rng=random.Random(1))))

go deeper

for a junior

Remember the shape: wait a little, then wait longer, then give up. Be able to say why retrying instantly in a loop is worse than waiting, and that the wait is usually doubled each time.

for a middle

Expect to write the formula, including the min that clamps it, and to explain jitter as breaking synchronisation between callers that failed together. Know that time.sleep blocks and asyncio.sleep is its coroutine counterpart.

for a senior

Show that you size the cap and the deadline from the caller's latency budget, measure elapsed time with time.monotonic, and never sleep while holding a lock or a connection. Be able to state the worst-case total wait of your own schedule.

for a principal

Own the numbers as a policy: what caps and deadlines apply to foreground versus batch paths, how retry behaviour is observable, and how you keep every service from inventing its own schedule with its own worst case.

## What the exponent buys you The delay schedule for attempt *n* is normally `base * 2 ** (n - 1)`: with a 200 ms base you wait 0.2, 0.4, 0.8, 1.6, 3.2 seconds. The point is not politeness, it is information. A failure that is going to clear in milliseconds — a dropped connection, a leader election, a brief queue — clears on the first short retry. A failure that is going to take a minute is not helped by a hundred immediate retries; those simply add load to something already unable to serve. Exponential growth lets one schedule cover both cases: cheap and fast when the problem is transient, quiet and patient when it is not. A constant delay cannot do that. One second flat is too slow for the blip and far too aggressive for a real outage, because the caller keeps arriving at a fixed rate for as long as the outage lasts. ## Why the ceiling Doubling is only sensible for a while. Left unbounded, attempt ten sleeps for a hundred seconds, which is almost never what anyone wants: the caller is either gone or the operation has been superseded. Clamping with `min(cap, base * 2 ** (n - 1))` turns the schedule into "grow quickly, then poll steadily at the cap". Pick the cap from the caller's tolerance, not from the dependency's recovery time — a foreground request that a user is waiting on may cap at a second or two, while an overnight batch flush can cap at a minute. ## Why the jitter This is the part candidates most often miss. Imagine a batch of workers flushing chat-transcript batches to an archive service. The service stalls for twenty seconds. Every worker fails at nearly the same instant, and with a deterministic schedule every worker sleeps 0.2 s, then 0.4 s, then 0.8 s — so they all retry *at the same three moments*. The recovering service sees a flat idle period punctuated by synchronised spikes, and each spike is large enough to knock it back down. The retries have accidentally become a clock that keeps everyone in phase. Randomising the wait breaks the phase. The common recipe is **full jitter**: compute the window, then sleep a uniform random value inside it, `random.uniform(0, window)`. Its expected delay is half the window, and it spreads arrivals across the entire interval. **Equal jitter** — half the window plus a random half — keeps a guaranteed minimum wait at the cost of a narrower spread, which is useful when you also want to be sure you are not retrying too early. **Decorrelated jitter** derives each delay from the previous one rather than from the attempt number, which spreads a long tail of retries more evenly. Any of the three is defensible; no jitter at all is not. Note that jitter also removes the pathological case where the schedule happens to line up with a periodic job on the other side. ## Attempts are the wrong bound on their own An attempt count says how many times you will call. It does not say how long the caller waits, and the caller's contract is expressed in time. Total latency is the sum of the per-attempt timeouts plus the sum of the sleeps, so "five attempts" can mean anything from a second to a minute depending on where the failures land. The robust shape is a deadline captured once with `time.monotonic` before the first attempt; before each sleep, check whether the deadline would be crossed and re-raise immediately if it would. `time.monotonic` is the right clock because it never jumps when the system clock is adjusted. ## Sleeping in the wrong place Two mistakes recur. The first is calling `time.sleep` inside a coroutine: it blocks the thread running the event loop, so every other task — including unrelated ones — stalls for the whole backoff. Inside `async def`, sleep with `asyncio.sleep`. The second is sleeping while holding something: a lock, an open connection, a database transaction. The backoff window is exactly when you should be holding nothing. ## Make it testable Keep the delay calculation in its own small function that takes the attempt number and returns a number. Then a unit test can assert the whole schedule — growth, the clamp, and that jitter stays inside the window — with a seeded `random.Random`, without a single real sleep. The wrapper that calls `time.sleep` should be the only part a test needs to stub.

  • What is the difference between full jitter and equal jitter, and when would you pick each?
    Full jitter sleeps a uniform value in `[0, window]`, so it spreads callers across the whole interval and has an expected delay of half the window. Equal jitter sleeps `window/2` plus a uniform value in `[0, window/2]`, guaranteeing a minimum wait but spreading arrivals over only half the range. Prefer full jitter when many callers fail together; prefer equal jitter when retrying too early is itself expensive.
  • Why is a total deadline a better bound than an attempt count?
    Because the caller's contract is stated in time, not in calls. Five attempts can take one second or sixty depending on where the per-attempt timeouts and sleeps land, so an attempt count gives you no latency guarantee. Capture a deadline with `time.monotonic` before the first attempt and abandon the retry when the next sleep would cross it; the attempt count then becomes a secondary safety limit.
  • What goes wrong if the backoff sleep happens while a lock is held?
    Everything waiting on that lock is now blocked for the entire backoff window, so one slow dependency becomes contention for unrelated work, and the effect grows with each doubling. Acquire late and release before sleeping: compute the delay, drop the resource, sleep, then re-acquire on the next attempt.

saying these in an interview costs you the question

  • Retries immediately in a tight loop with no delay
  • Uses one fixed delay for every attempt
  • Lets the doubling run uncapped into minute-long sleeps
  • Thinks jitter only matters for randomised testing
  • Calls time.sleep inside an async def coroutine
  • Bounds retries by attempt count but never by total time

context

resolving links…