You decide to put a short-lived in-process cache in front of Redis for a handful of extremely hot cache entries. How do you choose the TTL and the scope, and what failure modes does that introduce?

## What a near cache actually buys Every Redis hit is a network round trip plus work on a shared, single-threaded server. An in-process cache turns that into a hash lookup. The arithmetic is what makes it the strongest hot-key tool: - 40 instances × 5,000 rps on one key = 200,000 Redis GETs/s. - With a 1-second local TTL, each instance issues at most ~1 GET/s for that key → ~40 GETs/s total. That is a four-orders-of-magnitude reduction from a change that touches only the caching client. No resharding, no extra machines, no key renaming. ## Choosing the scope The first decision is *which* entries. Two failure patterns bracket the answer: - **Cache everything locally** and every process ends up holding a large slice of the keyspace: heap grows, GC pauses lengthen, and your consistency story gets worse across the board for entries that were never hot. - **Cache nothing locally** and you keep paying the network for keys that never change. The right scope is a **bounded LRU over measured-hot, read-mostly, small entries**. Bounded means a hard `maximumSize` (or byte-weight limit) — an unbounded map in front of a cache is a memory leak with extra steps. "Measured" means fed by the hot-key detection you already run, or a static allowlist of known globals (feature flags, config blobs, currency tables, top-N lists). Small matters because the entry is duplicated in every process. ## Choosing the TTL TTL is a **staleness budget**, and the budget is the product's, not the engineer's. Two rules: 1. **Staleness stacks.** Worst case, a value that was written to Redis just before you read it can be served from your local cache for the full local TTL, and the Redis copy itself may be up to its own TTL behind the source of truth. Total = local TTL + Redis TTL. Teams routinely forget the addition and are surprised by a 65-second stale window from a "5-second" local cache. 2. **The win saturates fast.** Going from no local cache to 1 second removes ~99.98% of the traffic in the example above; going from 1 second to 60 seconds removes almost nothing more but multiplies staleness sixty-fold. So start at the smallest TTL that solves the load problem — usually well under a few seconds — and only extend it if the numbers demand it. Some entries justify longer: an entry that changes on deploy only (a compiled ruleset) can sit locally for minutes, provided there is an invalidation channel or the deploy itself clears it. ## Failure modes **Per-instance divergence.** Instances refresh at different moments, so at any instant different instances hold different versions. Behind a load balancer, a user pressing refresh may see the new value, then the old, then the new — "flapping". If the UI shows a value that the user just changed, this looks like a bug. Mitigations: pin a user's session to an instance for a short window, bypass the local cache for read-after-write paths, or accept it explicitly for data where it does not matter. **Invalidation is hard.** Once a copy is in a process's heap, Redis cannot reach in and delete it. Options in increasing sophistication: rely on TTL expiry only (simplest, and correct if the TTL is inside the staleness budget); publish invalidation messages on a Pub/Sub channel that every instance subscribes to (delivery is best-effort — a disconnected subscriber misses the message, so keep the TTL as a backstop); or use the RESP3 client-side caching / tracking protocol, where the server itself notifies clients that a cached key changed (a distinct mechanism with its own semantics, and a topic in its own right). **Synchronized expiry.** Under steady traffic, entries populated at the same moment expire at the same moment. Across many instances that means a synchronized burst of Redis GETs, and if Redis has also expired the key, a synchronized burst of recomputes. Add jitter (±10–25%) to local TTLs. **Cold-start herd.** A rolling deploy, an autoscaling event, or a crash loop empties local caches. Every restarted instance repopulates from Redis at full request rate for its first TTL window. Usually survivable — Redis is the thing you are protecting and it is fast — but if the local cache is masking an already-marginal Redis, the deploy becomes the incident. Stagger rollouts; consider pre-warming the handful of global keys at startup. **Memory and GC.** Entries are duplicated per process, and in managed-runtime languages large object graphs promote to the old generation and lengthen pauses. Prefer caching the serialized form, or the parsed form if parsing is the actual cost — measure which. **Debuggability.** "Redis has the right value but production shows the old one" is a confusing incident. Expose the local cache's contents and age via a diagnostic endpoint, emit local hit-ratio and entry-age metrics, and offer a kill switch that disables near caching without a deploy. ## What not to near-cache Anything where a stale read is a correctness or safety failure inside your TTL: authorization decisions and permission sets, feature kill switches used to stop an incident (a 60-second local cache means your kill switch takes 60 seconds), quota and balance checks, and anything a user just wrote and will immediately read back. For kill switches specifically, choose a TTL you would accept as your incident-mitigation latency — often 1–5 seconds — and document it. ## How it composes A near cache stacks with everything else: it is layer 0 in front of Redis, and it reduces the load that key splitting or replica reads would otherwise have to absorb. In practice, try it *first* for a read-hot key, measure the remaining Redis traffic, and only then decide whether splitting or replicas are still needed.