Why does the price gap between RAM and disk, not any software limit, decide how much data an in-memory tier can hold?

## What a byte costs on each medium The media in a server are ordered by price per byte about as reliably as they are ordered by latency, and in the same direction. RAM is the most expensive storage in the machine; flash sits below it; bulk spinning disk is cheapest by a wide margin. The gap between RAM and bulk disk is commonly one to two orders of magnitude per byte. It moves with market conditions, module generation and capacity, but it has never been small. That single fact makes an in-memory component a different kind of budget line from a disk-based one: - a component that answers from memory buys RAM **in proportion to what it holds**; - a disk engine buys RAM in proportion to the part of its data that is hot, and disk in proportion to all of it. The second curve is far flatter as data grows, and that is the whole economics of the volatile tier in one sentence. ## The ceiling is a machine, not a setting Price is not the only constraint. A machine has a fixed number of memory slots, which produces effects people often miss: - the largest modules typically carry a premium per byte over mid-sized ones, so price per byte can rise as you fill a machine; - past the machine's maximum there is no more memory to buy at any price on that machine - only a larger machine, or another one; - bulk disk has no comparable wall at the capacities most systems care about. The practical consequence is that the capacity an in-memory tier can offer is bounded from outside the software. No setting relaxes it, because it was never a software limit. 'How much can we keep in memory' is really 'how much memory are we willing to buy, and does it fit in machines we are willing to run'. ## What that does to the shape of a design Because the ceiling is economic and physical rather than arbitrary, in-memory designs converge on one shape: **the tier holds a chosen subset, and something else holds the rest**. The medium does not choose the subset. It only guarantees that there is one. It is worth being precise about what the price gap does and does not bound: | question | decided by the medium's price? | |---|---| | how much data can be resident at once | yes, directly | | whether the design must choose a subset at all | yes, that is the ceiling's consequence | | which entries belong in that subset | no, that is a workload question | | what the component does once it fills | no, that is the store's own behaviour | The first two rows are the medium's business. The last two are answered elsewhere, and a candidate who knows the tier also knows where its argument stops. ## Where the price gap stops mattering The argument has a floor. If the data in question is a few gigabytes - a routing table, a set of live sessions, a day of counters - the difference between buying that much RAM and that much disk is a rounding error against the cost of the machine and the people running it. At that size the medium argument has no force in either direction, and the decision has to rest on something else: what the data is for, what depends on it, what the failure domain looks like. The gap also moves in a direction that is easy to miss. As a dataset grows, the cost of keeping all of it resident grows with it, while the cost of keeping only its hot part resident grows far more slowly. A design that started comfortably in memory can outgrow the medium's economics without anything else about it changing - the same code, the same access pattern, a bill that has quietly changed shape. ## Where stores differ Two variations matter here and should never be assumed away: - **Some stores in this class can keep colder values on a solid-state device**, fetching them back when touched. That shifts the ceiling outward and lowers the average price per stored byte, at the cost of a device access on the values that were moved. It changes where the ceiling sits; it does not remove it. - **Spreading data over several machines** raises total memory in proportion to the machines. The per-byte price of RAM is identical on each of them, so this buys capacity with hardware, coordination and a partitioned view of the data rather than with a better exchange rate. How that layout works is a separate subject; the economics of the medium are unchanged by it. Stated plainly: the software is not the thing saying no. The medium is, and it says no in currency.