One Slow Operation

## Two cost curves behind one call site An in-memory store answers a single-key read or write in microseconds, and that number barely moves as the keyspace grows: the server finds the key in its index and copies one value. Call that the **constant class**. A second class exists in every store of this kind and obeys a different rule — the server must touch many stored items before it can reply, so its **server service time** (what the store spends executing, as distinct from the **caller's wall time**, which also contains the network trip and any waiting) is proportional to how much is stored. The defining property of that second class is a mismatch the call site hides: - the **request** is a few bytes and looks exactly like a cheap one; - the **work** is set by data the caller never mentions; - the cost therefore **grows on its own**, with no code change to trigger an investigation. ## What the work is proportional to - **The member count of one entry.** Where a store understands structure — a map, a list, a set, a score-ordered collection — an operation that returns every member, or that combines, sorts or ranks a whole collection, walks every member and serializes each one. Ten members and ten million members are the same line of code at the call site. - **The key count of the whole keyspace.** An operation that must consider every key — a keyspace-wide listing, an administrative sweep — is proportional to the number of keys the store holds, not to anything in the request. - **The bytes written out.** Even when the walk itself is cheap, producing and buffering a reply of many megabytes occupies the server for the duration. What the cost is *not* proportional to is the argument. A request that names one key and asks for all of it is small however large "all of it" turns out to be. | Operation shape | What sets its server service time | What the request reveals about that | |---|---|---| | Single-key read or write | one index lookup and one value copy | the whole cost | | Returning every member of one entry | that entry's member count | nothing | | Combining, sorting or ranking a collection | member count plus the ordering work | nothing | | A keyspace-wide listing or sweep | the number of keys stored | nothing | ## The arithmetic that turns it into a production problem Suppose one request path returns every member of a collection that gains members steadily. At ten thousand members the call costs a few milliseconds and nobody notices; at a million it costs hundreds. Nothing in the deployment changed — no release, no configuration change, no traffic shift — and staging, where the collection is small, will never reproduce it. Inline arithmetic is the strongest evidence you can bring to a design review. If the server spends roughly a microsecond per member walked and serialized, a hundred thousand members is roughly 100 ms of server service time charged to one call. Against a typical operation of ten microseconds, that single call is the work of ten thousand ordinary ones. ## Stores differ in which expensive calls they even have This is the part most often got wrong by generalising from the one store someone has used: - Stores that keep values as **opaque bytes** and answer only single-key reads and writes have **no** whole-collection read, no server-side combination and no ranking — the server never inspects the value. Their expensive class is one very large value being copied out, plus any operation that touches the whole keyspace. - Stores that **understand structure** add the whole family of collection operations, and with them the possibility of a call whose cost is decided entirely by stored data. - Stores differ again in whether a **bounded** or **incremental** alternative exists for those operations — a range instead of everything, a cursor returning a slice per call. "Just use the incremental form" is advice that has to be checked against the store in front of you. Naming which of these the store is should come before any estimate of what a call costs. ## The habit this leaves you with For every call a request path makes to the volatile tier, ask one question: **is this call's cost decided by my request, or by what happens to be stored?** If the answer is "by what is stored", the call is in the expensive class and needs a bound — a range, a slice with a cursor, a limit the store enforces — before it ships, not after the collection grows. A cost that grows silently is worse than a cost that is merely high, because nothing ever prompts anyone to look.