Your service already runs the mapper's shared cache, and you are asked to add an application cache of assembled models. How do you decide, and what does running both cost?

## Ask what the read is actually paying for The two layers remove different costs, so the decision is empirical before it is architectural. Profile one slow read and attribute its time: - **Round trips and row fetching dominate** — many small lookups by identifier, a graph walked one association at a time. The mapper's own shared tier is the fit. It is invisible to your code and it invalidates itself when the mapper writes. - **Assembly dominates** — wide joins, aggregation, a result stitched from several queries, then projected and serialized. A row cache changes nothing here; the rows were cheap and the work was afterwards. Only caching the finished answer removes it. - **Neither dominates and the read is simply frequent** — the honest answer is often that no cache is warranted yet, and an index or a narrower projection is the cheaper fix. ## What the second layer costs 1. **A second invalidation surface.** The mapper's tier is maintained by the component doing the writing. An application entry is maintained by your write paths, every one of them, forever — including the ones added next quarter by someone who does not know the cache exists. This is the dominant long-run cost and it is paid in code that is easy to forget rather than in latency. 2. **Shape coupled to deploys.** Row data is governed by the schema; an assembled model is governed by code that ships independently. Entries written by the previous version outlive it, so the key needs a shape version and a rolling deploy needs thinking about. 3. **Ambiguous debugging.** A user reports stale data. With one layer there is one explanation; with two there are three, counting the interaction. Both layers need to be observable — hit rate, age, and a way to say which one answered — or incident time grows. 4. **Compounded staleness.** A miss in the upper layer can be filled from a hit in the lower one, so an assembled answer can be built from row data that was itself already old. The worst case is the sum of the two windows, not the larger of them. ## Deciding, in order | Question | If yes | If no | |---|---|---| | Is the dominant cost assembly rather than fetching? | An application entry is the only layer that can help | Prefer the mapper's tier or a better query | | Must the cached answer be shared across processes? | Favours an external application store | An in-mapper tier may suffice | | Can every write path be found and updated? | Two layers are maintainable | Keep one, and let it be the self-maintaining one | | Is a bounded staleness window acceptable for this data? | Caching is on the table at all | Do not cache; fix the read | ## If you run both, make their scopes disjoint The failure mode of two overlapping caches is a write that must invalidate an unpredictable set of keys in both. Avoid it by construction: - Let the mapper's tier own **by-identifier row reads** and nothing else. - Let the application cache own **expensive assembled answers**, keyed by the inputs that produced them. - Do not cache the same read in both. A read that hits above never reaches the mapper, so the lower entry only adds memory and another way to be stale. - Prefer dropping an application entry over rewriting it on a write, so the next read reassembles from whatever the lower layers currently hold, rather than freezing a value the write path composed by hand. ## The organisational half of the answer The durable cost of this decision is not latency, it is the standing obligation. A cache above the mapper is a piece of shared state your team now owns: its freshness rules, its key discipline, its shape versioning, its behaviour during deploys and during a store outage. The right question in the design review is not "will this be faster" — it will — but "who maintains the invalidation when the person who wrote this has moved on". Two things make the obligation survivable. Enforce the mechanism in one place rather than by convention: a single component that builds keys, populates after commit and drops on write, so no service reinvents it. And give the cache a bounded lifetime so that every mistake in it is self-correcting, at a cost you chose deliberately rather than one you discovered during an incident. Finally, keep the reversal cheap. A cache added behind a clean boundary can be turned off under load or during an incident and the system merely gets slower; a cache the code has come to depend on for correctness cannot. Design for the first.