Why must per-user request state leave the application instance once a second instance exists, and what does the move cost?

## Why the problem exists at all An application instance is a process, and a process's memory belongs to it alone. State a request handler puts in a field, a map or a framework's in-memory holder is readable by every thread of that process and by nothing else. Run a single instance and this is invisible: the same process handles every request from every user, so what one request wrote is there for the next. Run two, behind a load balancer that spreads requests, and a user's next request can arrive at a process that has never seen them. Nothing is broken — the second process simply has no record of the first one's work. That is the whole of the problem, and it appears the moment a deployment stops being one process, which for most systems is the moment they want to restart without downtime. ## The two exits There are only two shapes of answer, and they differ in what they make interchangeable. **Pin the user to the process.** Routing is configured so every request from one user goes back to the instance that first served them. The state stays in local memory: reads cost a memory access, nothing is serialised, and no extra component exists. **Move the state out.** Every instance reads and writes that state in a shared volatile tier all of them can reach. The instances become interchangeable, and the state becomes a thing with its own address, its own lifetime and its own capacity. | | Pinned to the instance | On a shared tier | |---|---|---| | Who can serve the next request | only one instance | any instance | | Cost of a read | a local memory access | a network round trip | | A deploy, crash or scale-in | takes that instance's state with it | leaves the state where it is | | Load balancing | constrained by who holds what | free to send a request anywhere | | New failure mode | losing one process affects its users | the tier sits on every authenticated request | Pinning's bill arrives on the most ordinary days — a rolling deploy, an instance replaced by the scheduler, scale-in after a quiet hour. Each of those takes a slice of the signed-in population's state with it. Affinity also fights load balancing: a heavy user stays on one process however busy that process gets. ## What the move actually costs Externalising does not make the application stateless. It moves the state onto a component that is honest about being stateful, and the bill is charged per request: - **A network call before any work.** Deciding whether a request may proceed now needs a call to the tier, and it sits in front of the work rather than beside it. The tier's latency, including its tail, is added to every authenticated request. - **Serialisation in both directions.** The state must be encoded to be stored and decoded to be used. On a store that treats the value as opaque bytes and hands them back untouched, that is the whole entry every time; a store that understands the value's structure may return only the part you asked for. A large share of this class does the former, so do not design assuming the latter. - **A second cost when the entry's deadline moves.** A fixed lifetime (counted from the write) leaves the entry alone while the user is active. An access-extended lifetime (pushed forward by each use) turns every request into a write as well as a read. - **A shared failure domain.** Every signed-in user now depends on one component, which is a capacity and availability conversation to have in advance with whoever operates it. ## Keeping it to one round trip 1. **Hold one user's state under one key**, so a request fetches it in a single operation instead of assembling it from several. 2. **Read it once per request** and pass it down, rather than re-reading in every layer that happens to want it. 3. **Keep the entry small**, because it crosses the network on every request — and on an opaque-value store it crosses whole. ## What an interviewer is listening for The junior answer is "the servers do not share memory", which is correct as far as it goes. The stronger answer names both exits and says why most systems chose the second: not because it is cheaper per request — it is not — but because it makes instances disposable, which is what deploys, autoscaling and failure recovery all depend on. The strongest answer then prices the choice out loud: a round trip in front of every authenticated request, a serialised payload each way, and a component whose bad minute belongs to everyone who is signed in.