What is a cache stampede (thundering herd) in a Redis-backed cache, and what exactly does the request flow look like in the moment a popular cached entry expires?
answer
- Herd size ≈ rps × recompute time
- Window opens at expiry, closes at first SET
- Positive feedback: slower source → wider window
- Three kinds: single key, correlated TTLs, cold cache
- Fixes: mutex, early refresh, stale, singleflight
basics
~20 sWhen a popular key expires, every concurrent request gets nil from GET and all of them recompute the same value against the database at once. The number of duplicate recomputes is roughly request rate times recompute latency, and the resulting slowdown makes the window longer.
solid answer
~60 sThe window opens the instant the key expires and closes when the first recomputed value is written back with `SET`. Every request arriving in between sees `nil` from `GET` and independently runs the expensive work. The size of the herd is arrival rate × recompute latency: 5,000 rps on one key with a 200 ms rebuild means about 1,000 simultaneous identical database queries. That is bad on its own, but the feedback loop is what causes outages — the database slows under the load, so recompute latency grows, so the window widens, so more requests join the herd. Client timeouts trigger retries, adding more. Three flavours worth naming: **single-key expiry** (one hot key), **correlated expiry** (many keys written together with the same TTL expiring together), and **cold cache** (restart, flush, failover to an empty replica, or mass eviction under `maxmemory`) where everything misses at once. The Redis-side toolkit is a `SET NX PX` mutex around the recompute, probabilistic early refresh, serving a stale copy while one worker rebuilds, and coalescing duplicate work inside each process.
go deeper
Describe the flow: key expires, many requests miss at once, all of them hit the database with the same query.
Quantify it (rate × recompute time), explain the feedback loop, and name the standard fixes including a Redis mutex on the miss path.
Distinguish single-key, correlated-TTL and cold-cache stampedes, and match each to the right countermeasure, including bounded waiting so the herd does not just move into your thread pool.
Frame it as load-amplification risk: define which entries may ever be recomputed on the request path, set concurrency limits and degradation behaviour, and design for cache loss as an expected event.
## The mechanism, step by step A cache-aside read is: `GET key` → hit, return; miss → compute the value from the source of truth → `SET key value EX ttl` → return. Nothing in that sequence coordinates concurrent callers. So when a key expires: 1. `t=0` — the TTL elapses, Redis no longer returns the key. 2. `t=0…` — every arriving request gets `nil` and starts the expensive work: a slow SQL aggregate, a fan-out to three services, a template render, a machine-learning scoring call. 3. `t=R` (recompute latency) — the first finisher writes the value back; from then on requests hit again. Duplicate work ≈ **arrival rate × R**. At 5,000 rps and R = 200 ms, ~1,000 concurrent identical queries land on a database sized for the *cached* traffic, i.e. a trickle. Nothing about Redis caused this; Redis simply stopped hiding the real load for R seconds. ## Why it escalates instead of just being wasteful The dangerous property is positive feedback: - The database queues; each query now takes 2 s instead of 200 ms. - Because R grew tenfold, the miss window grew tenfold, so ten times more requests join the herd. - Application threads and connection-pool slots are all parked waiting, so unrelated endpoints start failing too. - Clients time out and retry, adding load that is pure waste — the original work is still running. That is congestion collapse: a system that was comfortably serving 5,000 rps from cache cannot serve 5 rps from the source, and it will not recover on its own until traffic drops or something sheds load. ## Three flavours **Single-key expiry.** One very hot entry. The classic case, and the one the mutex and early-refresh techniques target. **Correlated expiry.** A batch job or a deploy writes ten thousand entries within a second, all with `EX 3600`. An hour later they all expire within the same second, and the herd is across many keys — a mutex per key does not help nearly as much, because there is no shared work to deduplicate. The countermeasure is spreading expiry times when writing them, which belongs to how you choose TTLs in the first place. **Cold cache.** Redis restarts without persistence, someone runs `FLUSHALL`, a failover promotes a replica that is behind, or memory pressure under `maxmemory` evicts a large fraction of the keyspace at once. Now *everything* misses simultaneously. No per-key trick saves you; you need admission control — rate-limit or queue the recomputes, warm critical keys before taking traffic, and be able to shed or degrade. ## The mitigation toolkit (all Redis-side) 1. **Mutex on recompute** — `SET lock:<key> <token> NX PX <ttl>`. Exactly one caller wins and rebuilds; the losers wait briefly, serve stale, or degrade. Simple, effective, and it introduces a new question: what should the losers do? 2. **Probabilistic early expiration (XFetch)** — store the value together with its computed cost and a logical expiry time, and let each reader independently decide, with rising probability as expiry approaches, to refresh *before* the key is gone. Usually one reader refreshes, and nobody ever sees a miss. 3. **Serve stale while revalidating** — set the physical Redis TTL longer than the logical freshness deadline, so `GET` always returns something. One caller refreshes; the rest immediately return slightly stale data. This is what makes the mutex non-blocking, and it also protects you when the source of truth is down. 4. **Request coalescing (singleflight)** — inside each process, collapse concurrent requests for the same key onto one in-flight computation. Free, no network, and it removes the largest multiplier (threads per instance) before you even talk to Redis. 5. **Negative caching** — cache "not found" for a short TTL so a missing entity does not stampede on every request. 6. **Async refresh** — a background worker refreshes hot entries on a schedule so the request path never recomputes at all. Great for a small, known set of expensive entries; unmanageable for a large keyspace. In practice a mature service layers them: singleflight in-process, a Redis mutex across processes, stale-while-revalidate so waiters never block, and jittered TTLs so herds do not synchronize. ## What does *not* fix it Lengthening the TTL only makes the stampede rarer and larger, and increases staleness. Removing the TTL turns your cache into a memory leak with unbounded staleness. Adding Redis capacity is beside the point — Redis was never the bottleneck; the source of truth behind it was.
- Why does simply increasing the TTL not solve a stampede?A longer TTL makes expiry rarer but does not change what happens at expiry — the same herd forms, and by then the value is even more popular, so the herd may be bigger. It also increases staleness and, for correlated writes, keeps the synchronized-expiry problem intact. You have to change the *miss path*, not the interval between misses.
- How is a stampede after a Redis restart different from a stampede on one hot key expiring?A hot-key stampede is many requests for the *same* value, so deduplication (a mutex or singleflight) collapses it to one recompute. A cold cache is many requests for *different* values, so there is nothing to deduplicate — every recompute is genuinely needed. That calls for admission control instead: bound the concurrency of recomputes, pre-warm critical keys, and shed or degrade rather than queue.
- Where does a stampede usually show up first in your telemetry?A sudden spike in source-of-truth concurrency and latency alongside a dip in cache hit ratio, with application thread pools and database connection pools saturating at the same moment. Redis itself typically looks healthy — normal latency, normal CPU — which is a useful signal that the cache is not the bottleneck, the thing behind it is.
A single ferry ticket booth serving a queue of 1,000 people: while the one clerk is looking up an answer, everyone else asks the same question instead of waiting for the answer to be posted.
saying these in an interview costs you the question
- Proposing a longer TTL as the fix
- Removing the TTL entirely so the key 'never expires'
- Believing Redis serialises concurrent misses for the same key on its own
- Blaming Redis performance when the saturated component is the database behind the cache
- Assuming a per-key mutex also protects against a cold cache, where every request needs a different value