Two components one hop away serve the same 2 MB value, one from RAM and one from a warm disk engine - why is the latency gap small?

## Two different costs wearing the same word The latency of a read is really two costs added together. 1. **Access time** - what it costs to begin getting the bytes: a memory access, a device access, a lookup on a server, the first packet of a round trip. It is roughly a constant per operation and barely depends on how much is being read. 2. **Transfer time** - what it costs to move the bytes once they are moving: bounded by memory bandwidth, device throughput and, when a network is in the path, the link rate. It is proportional to the payload. Every memorable memory-against-disk ratio is a ratio of **access time**. That is why it shows through clearly on a small read and quietly stops mattering on a large one: for a large payload, both components spend most of their time on a cost that is nearly identical for both. ## The arithmetic on a two-megabyte value Take the same two-megabyte value served two ways, each one network hop from the caller. | cost | in-memory component | warm disk engine | |---|---|---| | access on the server | around a hundred nanoseconds | resident page: comparable; cold page: tens to hundreds of microseconds | | reading the payload locally | memory bandwidth, gigabytes per second | memory or device bandwidth, gigabytes per second | | moving it across the network | about 1.6 ms on a ten-gigabit link | about 1.6 ms on the same link | Serialising two megabytes onto a ten-gigabit link takes on the order of 1.6 milliseconds before anything else is counted. Against that, an access-time difference measured in microseconds is a few percent of the total. Even a **cold** device access of several hundred microseconds is a minority share. The ratio the slogan promises has collapsed toward one - not because the media became equal, but because the question stopped being about them. ## Where the crossover sits A rough way to hold it: the medium's advantage shows through while the payload is small enough that **moving** the bytes costs less than **beginning** to move them. With access-time differences in the tens to hundreds of microseconds and link rates in gigabits per second, that crossover lands somewhere around tens of kilobytes on typical hardware. Below it, per-operation costs dominate and the medium argument is live. Above it, the bytes dominate and it is not. The exact number moves with hardware; the structure does not. This has a practical edge for anyone comparing components: - a benchmark that reads small entries and a production workload that reads large values are measuring different regimes; - a ratio quoted from the first will not survive in the second; - when payload sizes vary widely, aggregate latency is dominated by the large reads even if most requests are small, so an average ratio describes neither regime; - headroom on the link, not the medium, is what a large-value workload actually consumes. ## What this does and does not license It does not say large values belong on one component rather than the other, and it says nothing about what a large value does to the other callers sharing a component - that is a separate subject with its own answer. It says one thing precisely: **on large payloads, the medium is not the reason to prefer one component over the other**. If a design is choosing between them for large objects, the honest grounds lie elsewhere - who holds the authoritative copy, what happens when the component is unavailable, what the bandwidth and the storage actually cost. The corollary is useful in reviews. When someone justifies a component on the media ratio, ask how large the values are. If they are kilobytes, the argument is at least in the right regime and the remaining question is the baseline. If they are megabytes, the ratio has been quoted from a regime the workload is not in, and the discussion should move to bandwidth and ownership. ## What varies - **Where the components sit.** The collapse argument assumes both pay the same network. If one is inside the calling process and the other is across a link, the large-payload comparison is memory bandwidth against link rate, and that is not close. - **How a change to a large value is made.** Stores in this class differ in whether part of a value can be altered on the server or whether the whole value must be moved out and back. Where it is the latter, every update to a large value is a large-payload transfer in both directions, which puts the workload firmly in the regime described here. - **The engine's residency.** A warm engine's access is a memory access, a cold one's is a device access. Both are small against a multi-megabyte transfer, which is what makes this argument unusually robust: it holds whether the engine is warm or not.