Only 5 million of a store's 200 million entries, all copies of database rows, are read hourly — which number sizes it?

## Two numbers that both get called "the size" Every capacity conversation about an in-memory store opens with two numbers, and they are usually an order of magnitude apart: - **Total data** — everything that exists and could conceivably be held: every row, document or object in the system of record. - **The working set** — the distinct **entries** actually touched inside a window you name: the last hour, the last day, one checkout flow. An *entry* is the key, the value and whatever bookkeeping the store keeps for it, taken together; the working set is counted in entries and then converted to bytes, not guessed as a share of the total. | The number | What it counts | When it is the right one to size for | |---|---|---| | Total data | Everything that exists | The store holds the only copy, or the whole set genuinely must be resident for a latency target | | The working set | Distinct entries touched in a chosen window | A source of truth exists elsewhere, so an entry that is absent is re-read rather than lost | ## Why the working set is the number here In the situation described, the rows exist in a database. That single fact decides the arithmetic. The worst thing that happens to an entry the store never held is that a caller waits for one read against the database and the entry is written on the way back. So memory spent holding entries nobody touches buys nothing measurable: it is not faster, not safer, and not cheaper. The two ways of getting it wrong are not symmetric: 1. **Sizing for total data** over-buys. The cost shows up as a bigger machine, a longer restart, and a larger blast radius when the node is lost — never as a wrong answer. 2. **Sizing below the working set** under-buys, and the workload tells you so: at flat traffic, the share of reads served from the store falls, because the tier can no longer hold one window's worth of touched entries at once. That asymmetry is why the working set is the *starting* number and the miss rate is the correction. ## When the arithmetic inverts A store of this class is not always holding copies. The same component is routinely asked to hold state that exists nowhere else — sessions between application instances, leases, quotas, deduplication markers. For that content: - there is no source to re-read, so an entry the store does not hold is **data loss**, not a miss; - "the working set" is not a meaningful reduction, because every live entry is one you must still be able to produce; - the number to size for is total live data, and the honest lever is how long entries stay alive, not how many of them you hold. A tier that mixes both kinds of content inherits the stricter rule for the part that has no source of truth. Deciding which content deserves to be held at all is a separate design question; the sizing rule only asks whether a source exists. ## Choosing the window, and how the window goes wrong The working set is not a property of the data — it is a property of the data *and a window*. Two traps follow: - **A window too short** flatters the number. Ten minutes of traffic on a site with a daily rhythm counts a fraction of what a day touches, and the tier under-performs outside the window you measured. - **A sweep inflates the number.** A nightly job that reads every entry once makes the distinct count over twenty-four hours equal to total data, while telling you nothing about the interactive traffic you are actually sizing for. Measure the window that serves the latency you care about, and treat a full sweep as its own workload. The working set also moves in bytes without moving in entries: if each stored value grows, the entry count is unchanged and the memory required is not. ## What varies between stores The concept is universal; the surrounding machinery is not. - Some stores expose a **memory ceiling** you configure; on others no such setting exists and the machine or container limit is the only bound, which makes the sizing decision a node-choice decision instead. - What a store counts as the cost of one entry differs — its own bookkeeping, how the value is laid out, and how the allocator rounds a request all differ between implementations, so the per-entry figure must come from a measurement on your own data rather than from a number you remember. - Some stores in this class offer no way to observe which entries were touched, so the working set is inferred from the miss rate rather than read off directly. The defensible answer in an interview is therefore not a number but a shape: *the working set, measured over a named window, converted to bytes by a measured cost per entry, and checked against the miss rate afterwards* — unless the tier is the only copy, in which case it is total live data.