A page nginx proxies takes 400 ms to generate and receives about 2,000 requests per second. Explain how an nginx microcache built on `proxy_cache_valid 200 1s` absorbs that load, and what still lets a burst of requests reach the upstream each time the entry expires.

## The arithmetic At 2,000 requests per second with 400 ms of generation time, an uncached origin needs roughly 800 requests in flight simultaneously (Little's law: 2000 × 0.4). Almost no application server pool is sized for that, so the queue grows, latency climbs, and the pool collapses. Cache the response for **one second** and, for a single hot URL, the origin sees about one request per second — a 2,000-fold reduction — while no client ever sees data more than a second old. That is the whole idea of microcaching: it is not aimed at content that is safe to cache for ten minutes, it is aimed at content nobody thought was cacheable at all. ```nginx proxy_cache_path /var/cache/micro keys_zone=micro:10m max_size=1g inactive=1m use_temp_path=off; location / { proxy_pass http://app; proxy_cache micro; proxy_cache_valid 200 1s; proxy_ignore_headers Cache-Control Expires Set-Cookie; proxy_hide_header Set-Cookie; } ``` The `proxy_ignore_headers` line is not optional in practice: a dynamic page almost always declares itself uncacheable and sets a session cookie, so without it nothing is stored. And because you are ignoring `Set-Cookie`, you must hide it — otherwise the stored copy replays one visitor's session identifier to everyone. Microcaching is therefore only safe on URLs that render identically for every anonymous visitor; logged-in traffic must be excluded before it reaches this location. ## The gap at the expiry instant Once per second the entry becomes stale. nginx's default behaviour for a stale entry is to proxy the request — and it does that for *every* request that arrives while the refresh is in flight. With 2,000 rps and a 400 ms regeneration, that is roughly 800 simultaneous upstream requests, once a second, for exactly the URL you were protecting. The cache flattens the average and leaves a spike. Two directives are relevant, and they cover different cases: - **`proxy_cache_lock on;`** allows only one request at a time to populate a **new** cache element — one that is not stored yet. Others wait, bounded by `proxy_cache_lock_timeout` (default 5s) and released early by `proxy_cache_lock_age`, after which another request is allowed to try. This is the cold-start case: a fresh deploy, an evicted key, the first hit after a purge. It does **not** apply to an entry that exists and has expired. - **`proxy_cache_use_stale updating;`** covers the expiry case. While one request is updating an expired entry, everyone else is served the stale copy and logged as `STALE`/`UPDATING`. Combined with **`proxy_cache_background_update on;`** (nginx 1.11.10 and later), the refresh happens in a background subrequest so even the triggering client gets an immediate stale response rather than waiting 400 ms. Together: ```nginx proxy_cache_lock on; proxy_cache_lock_timeout 5s; proxy_cache_use_stale updating error timeout http_500 http_502 http_503 http_504; proxy_cache_background_update on; ``` The extra `error timeout http_5xx` parameters buy something beyond the herd: when the origin is failing outright, nginx keeps serving the last known-good copy instead of propagating the failure. For a one-second microcache that window is short, but during a deploy or a database stall it is often the difference between a degraded page and an error page. ## What it costs - **Staleness.** Bounded by the validity, so a second — usually acceptable for a homepage or a listing, never for a checkout confirmation. - **Applicability.** Only URLs with real concurrency benefit. A long tail of URLs each getting one request per second gains nothing and just churns the cache; keep `inactive` short so the tail evicts itself. - **Correctness risk.** Everything in the personalisation discussion above applies with less margin for error, because you deliberately disabled the header that would have protected you. - **Observability.** Watch the distribution of `$upstream_cache_status`. Healthy microcaching is overwhelmingly HIT with a thin trickle of UPDATING; a visible band of EXPIRED means the stale-while-updating path is not configured, and BYPASS means your exclusion conditions are firing more than you thought. ## When not to reach for it If the same content can safely live for a minute, cache it for a minute — microcaching is the answer when the content genuinely changes constantly and the load is concentrated. If personalisation makes a shared copy impossible, the fix is to split the page: cache the anonymous shell and fetch the personalised fragment separately, rather than pushing a per-user document into a shared store.