How should a JWT verifier cache and refresh issuer keys so rotation causes no outage?

## Where this sits in validation Before a signature can be checked, the verifier must answer "which key?". For asymmetric signing the answer comes from the issuer's published key set, and the token's `kid` header names which entry to use. Everything interesting here is operational: the correctness rule is trivial (use a key you trust), but naive implementations produce either outages during rotation or traffic storms against the issuer. ## The cache is not optional Fetching keys per request would put a network call in front of every authenticated request — latency, and a hard dependency on the issuer being up. So the key set is cached in memory. Two consequences follow immediately: the cache can be *stale* when the issuer rotates, and the cache can *stampede* when many threads discover staleness simultaneously. ## Refresh on unknown kid, not on a timer alone Polling on a fixed interval is a poor primary strategy: too slow and rotation breaks authentication for up to one interval; too fast and you hammer the issuer forever to detect a change that happens quarterly. The event-driven rule is better — when a token presents a `kid` that is not in the cached set, that is the signal to refresh. Combine it with a modest background TTL so a long-idle process does not hold ancient keys. ## Guarding the refresh — the part candidates miss "Refresh when the `kid` is unknown" is exploitable as written. An attacker sends unsigned garbage tokens with random `kid` values; each one is a cache miss; each miss triggers a fetch. You have built a request amplifier pointed at your own identity provider. The guards: - **Rate limit / minimum refresh interval.** At most one refresh per N seconds regardless of how many misses occur. - **Single-flight coalescing.** Concurrent misses join one in-flight fetch instead of issuing N fetches. - **Negative caching.** Remember recently-seen unknown key identifiers for a short period and reject immediately without a fetch. - **Bounded parsing.** Cap the size of the fetched document and the number of keys accepted, so a hostile or broken response cannot exhaust memory. ## Serving during refresh While a refresh is in flight, keep answering from the existing cache. Tokens signed with keys you already hold must not queue behind a network call. Only requests whose `kid` is genuinely unknown wait — or, better, fail fast and let the client retry once the refresh lands. ## What makes rotation non-disruptive Rotation is a two-sided contract, and the verifier can only hold up its half: - The issuer publishes the new public key **before** it starts signing with it, so verifiers can learn it in advance. - Both keys remain published during an overlap window at least as long as the maximum token lifetime, so tokens signed with the outgoing key still verify. - Only after the last token signed with the old key has expired is that key withdrawn. The verifier's half: never assume a single key, always select by `kid`, tolerate several keys of the same type in the set, and re-read the set rather than pinning one key at startup. A verifier that caches "the" public key at boot will break at every rotation. ## Fetch integrity The key set must be retrieved over TLS from a URL that is part of your configuration or derived from the issuer's own discovery document — never from anything the token supplies. Validate the certificate chain; TLS is the only thing that binds the fetched keys to the issuer. Pin the issuer's host, and apply a connect and read timeout so a hung fetch cannot pile up threads. ## Failure behaviour If the key set cannot be fetched and no cached key matches the token, reject. The tempting failure modes are both wrong: skipping verification because keys were unavailable is an authentication bypass, and clearing the cache on a failed fetch destroys your ability to keep serving during an issuer outage. Keep the last good key set and prefer stale-but-trusted over empty. Also design for the issuer being down at process start. A cold cache plus an unreachable issuer means no authentication at all, so retry with backoff, expose the cache state in health signals, and consider seeding from a configured fallback for critical services. ## Observability Emit metrics for cache hits and misses, refresh attempts and failures, key-set age, and rejections due to unknown `kid`. A rising unknown-`kid` rejection rate is the earliest signal that a rotation is happening — or that someone is probing you. Log the `kid` and issuer on failure; never log the token itself. ## Interview framing Give the mechanism in one line — cache, select by `kid`, refresh on miss — then immediately name the three hazards that separate a working implementation from a naive one: refresh storms, blocking on refresh, and failing open when keys are unavailable.