How would you set and verify a maximum-staleness budget for a data-access layer's cache, and why is hit rate the wrong headline number?

## Hit rate answers the wrong question Hit rate is the number every cache reports and the number most teams put on the dashboard. It measures **work avoided**: the fraction of lookups that did not become a database read. It says nothing whatsoever about whether the values handed back were still true. A cache whose invalidation has been broken since the last deployment reports a *better* hit rate than a correct one, because nothing is being removed. As a headline it is actively misleading: it goes up as correctness goes down. The number that describes the risk is the **maximum staleness**: the worst-case time between a row changing and the last cached copy of it disappearing. That is the quantity a reader of your API cares about, the quantity an incident review will ask for, and the quantity you can actually design against. ## Where the budget comes from A staleness budget is not chosen from a menu; it is derived from what happens if someone acts on the old value. Work per dataset, not per cache: - What decision is made from this data? Displaying a description tolerates minutes. Deciding whether a request is authorised, or whether an account has funds, tolerates nothing — that data belongs on a live read. - Who notices, and how fast? A user who has just edited a value and is looking at the result notices instantly; a nightly report notices never. - Is the wrongness recoverable? A late-updating page corrects itself. A message sent, a payment taken, or an entitlement granted on an old value does not. - What does the write rate imply? For rarely-written data almost any budget is safe. For hot rows, the budget effectively decides how much of the cache is wrong at any moment. The output is a stated number per dataset — "at most N seconds" — and a decision for anything that cannot name one: do not cache it. ## Where the delay comes from The achievable bound is set by the weakest link in the invalidation path, not by the intent of the design. Wherever a removal can be missed outright, the bound falls back to the entry's maximum age: | Contribution | Typical size | What removes it | |---|---|---| | Delay between commit and the removal running | small, unless it is tied to method exit rather than commit | tie the hook to the commit | | Propagation to other instances | network and queue delay | one shared copy instead of per-instance copies | | Removals that never arrive at all | unbounded until something else clears the entry | a bounded maximum entry age as the backstop | | Writes that bypass the layer entirely | unbounded | ownership of the write path, or a change signal | The last two rows dominate most real systems, which is why a budget expressed as a bounded entry lifetime is usually honest, and one expressed only as an invalidation usually is not. ## Verifying the number instead of asserting it A staleness budget nobody measures is a comment. Three measurements make it real: 1. **Convergence probes.** Write a marker row on a schedule, then read it back through the normal path on every instance, and record the time until all of them return the new value. Chart the maximum, not the mean — the tail is the budget. 2. **Divergence sampling.** Compare a random sample of served values against fresh reads and alert on both the rate of mismatches and the age of the worst one found. 3. **Invalidation flow counters.** Count removals issued and removals applied per instance. A ratio that changes after a deployment is the earliest warning that the design has quietly stopped working. Alert on the tail of the convergence probe crossing the stated budget, not on hit rate moving. ## The judgment a lead actually owns Two decisions cannot be delegated to configuration. The first is which datasets are **ineligible** — the ones where an old value produces an irreversible action — because that decision has to survive people later trying to speed those paths up. The second is who owns the budget when the writers live in another team: a cache whose correctness depends on another team's write path is a coupling that must be written down, given a signal, or given up. Absent that, the budget silently becomes "however long until someone redeploys", which is not a number anyone can defend after an incident.