A cache-aside key backing a hot database query expires, and within the same millisecond thousands of concurrent requests for that key all miss the cache. Describe two distinct mechanisms an application can use to stop this from turning into a 'thundering herd' that overwhelms the database.

## What a thundering herd is A **thundering herd** (also called a **cache stampede**) happens in cache-aside when a single hot key — or many keys together — transitions from cache-hit to cache-miss at the same instant, and every one of the many concurrent requests that were relying on that key independently discovers the miss and independently starts the exact same expensive fallback: 1. query the database, 2. get the same result, 3. write it back into the cache. If a key backs a query that normally serves thousands of requests per second entirely from cache, the instant it expires you get a burst of duplicate, simultaneous, identical database queries — the database sees load it was never sized for, often causing latency spikes or timeouts that cascade into further retries and make the spike worse. ## The first mitigation — a per-key lock The first common mitigation is a lock (mutex) around the reload for a given key. When a request misses the cache, before querying the database it attempts to acquire a short-lived lock specific to that key (e.g. a Redis `SETNX` with its own short TTL as a safety valve against a crashed lock-holder). Only the request that wins the lock actually queries the database and repopulates the cache; every other concurrent request that loses the lock either - waits briefly and retries the cache read (expecting the winner to have populated it by then), or - falls back to serving a slightly stale previous value if one is retained. This collapses N concurrent database queries down to exactly one, at the cost of added latency for the requests that have to wait, and at the cost of building and operating the locking mechanism itself, including handling a lock-holder that crashes or is slow. ## The second mitigation — request coalescing The second common mitigation is **request coalescing**, also called **single-flighting** or **in-flight deduplication**. Instead of coordinating through the cache with an explicit lock, the application process itself tracks, in memory, whether a database load for a given key is already underway; if a second concurrent request for the same key arrives while a load is already in flight, it doesn't start a second database query — it simply attaches to the result of the one already running and receives the same value when it completes. This achieves the same "only one database query per stampede" outcome as locking, but: - it's implemented purely in application memory (Go's `singleflight` package and similar libraries in other languages are canonical implementations) rather than via a distributed lock, so it's cheaper and simpler within a single process; - it only dedupes requests landing on the same process — a fleet of many app instances still needs a cross-instance mechanism (like the lock above) to fully collapse the herd. ## A third, complementary technique A third, complementary technique is **probabilistic early recomputation** (sometimes called early expiry with jitter): instead of waiting for a hard expiry and then having everyone discover the miss at once, each read near the end of a key's TTL computes a small, increasing probability of proactively refreshing the value early, so that reloads get smeared across a window before the hard deadline rather than synchronized to the exact expiry instant. This avoids the herd from ever forming in the first place, at the cost of some extra, usually-redundant early reloads and a bit of algorithmic complexity in the read path. ## Combining them in practice A concrete example: a news site caches the rendered homepage under one key with cache-aside; if that single key expires during a traffic spike, every concurrent visitor triggers an identical, expensive render-and-database-query at once. Production systems facing this typically combine a per-key lock (so only one process actually re-renders) with in-process request coalescing (so multiple threads/requests within that one process share the single in-flight render) — the lock solves the cross-instance duplication, coalescing solves the within-instance duplication, and together they turn thousands of duplicate database queries into exactly one. The main trade-off across all these mechanisms is added latency and complexity for the unlucky requests that arrive during the reload window versus the alternative of simply accepting the database load spike; for genuinely hot keys, that trade is almost always worth it.