Your session API keeps sessions in local process memory - which hosting tiers does that choice rule out, and why?
answer
- how long does one instance live
- who decides that, you or the platform
- a second instance breaks it first
- deploys erase memory routinely
- cache in memory, record in a store
basics
~20 sLocal memory lasts exactly as long as one instance, so it rules out every tier where the platform, not you, decides when an instance goes away - most sharply an event-driven runtime, and in practice any tier that replaces instances on deploy.
solid answer
~50 sThe real question is not whether a tier allows memory - they all do - but how long one instance lives and whether anything routes a given user back to it. On a machine you rented, the process survives until you replace it, so local sessions mostly work until you run more than one machine. On a managed container platform or a fully managed application tier, instances are replaced on deploys, scale-in and host maintenance, so sessions survive an afternoon and not a release. On an event-driven runtime the platform creates and discards instances on its own schedule and nothing routes a user back to a particular one, so memory there is a cache at best. The usual fix is to move the session into a shared store and keep memory only as a cache with a cheap miss.
go deeper
Recall that anything in a process's memory disappears when that process does, and that on most hosting tiers a deploy replaces every process. Sessions therefore belong in a shared store.
Explain the mechanics: instance lifetime differs per tier, deploys and scale-in end instances routinely, and a second instance breaks local sessions even while every instance is healthy.
Demonstrate judgment about which state is safe in memory and which is not, what a correct miss path looks like, and why request-scoped routing tricks make a cache hit likelier without making anything durable.
Set the rule that decides this for many services: which categories of state may live in a process at all, and what the team must provide - a shared store and a cheap miss path - before any service is allowed onto a tier whose instance lifetime it does not control.
## "Stateless" is a statement about instance lifetime Candidates often treat statelessness as a property of code. It is more useful to treat it as a question about the tier: **how long does one instance live, who decides that, and does anything route a given user back to the same one?** Local memory is not forbidden anywhere. It simply expires when the instance does, and the four hosting rungs differ enormously in who controls that moment. ## How long an instance lives on each tier | Tier | Who decides when an instance ends | What that means for local memory | |---|---|---| | Plain machine | You do, mostly - plus failures and reboots | Survives for as long as you leave the machine alone, but only on that one machine | | Managed container platform | Mostly the platform - deploys, scale-in, host maintenance | Lives for the life of one instance, which is typically one release | | Fully managed application tier | The platform, on the same kinds of events | Same shape: a deploy is the common eraser | | Event-driven runtime | The platform, on its own schedule | A cache at best; an instance may be reused, and may equally vanish between two requests | Note the qualification on the last row. An event-driven instance is often reused for consecutive work, which is precisely what makes this trap subtle: a warm cache appears to work in testing and then does not, because reuse was never a promise. ## What empties local memory even when nothing fails 1. **A deploy.** In the usual rolling-replacement model every running instance is replaced with a new one, so everything held in memory goes with it. This is the most frequent cause and the least dramatic. 2. **Scaling in.** When demand falls, instances are removed. Whichever user's session lived on the removed instance loses it. 3. **Host maintenance.** The provider replaces hosts underneath you on every tier; the workload is moved, and the new copy starts empty. 4. **Ordinary routing.** With more than one instance behind a single address, consecutive requests from the same user need not reach the same instance at all - so the session can be "missing" while the instance holding it is perfectly healthy. Point four is worth dwelling on, because it is the one that bites first. Local sessions usually break not when an instance dies, but the moment there is a second instance. ## The fix, and its shape - **Move the record out.** Put the session in a store shared by every instance, and let any instance serve any request. - **Keep memory as a cache, never as the record.** A cache miss should be a slightly slower request, not a lost session. - **Make the cost of a miss cheap and bounded**, because on the smaller rungs misses are routine rather than exceptional. - **Do not reach for pinning a user to one instance as the fix.** It makes the local hit more likely; it does not survive the instance going away, and it turns an even load into an uneven one. - **Check start-up assumptions too.** State is not only sessions: a warmed lookup table, an accumulated counter or a deduplication set held in memory has exactly the same lifetime, and the same failure. ## Where in-memory state is still reasonable It is entirely reasonable within the scope of a single request or unit of work - parsing, buffering, accumulating a result - because that scope cannot outlive the instance by definition. It is also reasonable as a cache in front of a shared store on any tier, provided the miss path is correct and the code never treats the memory copy as authoritative. What is not reasonable on the tiers where the platform owns the lifetime is anything whose loss is visible to a user or corrupts a total: a login session, a partially assembled upload, a counter that is only ever incremented in memory. ## What an interviewer is listening for - Turning the question into instance lifetime and routing, rather than answering "it is stateful, that is bad". - Naming a deploy - not a failure - as the routine eraser. - Knowing that a second instance breaks local sessions before any instance dies. - Being able to say where memory is still the right place, so the answer is judgment rather than a slogan.
- The team proposes routing each user back to the same instance instead of moving the state out. What does that buy, and what does it not?It raises the local hit rate, which can be worth having for a cache. It does not make the state durable: the instance still goes away on a deploy, a scale-in or a host replacement, and the session goes with it. It also concentrates load unevenly across instances.
- Where is in-memory state still perfectly reasonable on a tier that recycles instances?Within one request or unit of work - buffering, parsing, accumulating a result - since that scope cannot outlive the instance. Also as a cache in front of a shared store, as long as a miss is merely slower and the code never treats the memory copy as authoritative.
In-memory state on a tier that recycles instances is a note left on a hot desk: it is there until someone else sits down, and nobody promised you the same desk tomorrow.
saying these in an interview costs you the question
- Calls a service stateless because it owns no database
- Expects local memory to survive a deploy
- Thinks pinning users to one instance removes the need for a store
- Treats an in-memory cache and the system of record as interchangeable
- Assumes instances are replaced only when something fails