What is session affinity (sticky sessions) in a load balancer, how is it typically implemented, and what breaks when the backend a client is pinned to becomes unavailable?

## What sticky sessions are, and why they exist Session affinity, commonly called 'sticky sessions,' is a load-balancing policy that overrides the normal per-request distribution algorithm to instead route every request from a given client to the same backend instance for the duration of their session. It exists because many applications, especially older or simpler ones, hold session state in the memory of the process that first handled a client: - a logged-in user's session object, - the contents of a shopping cart, - an in-progress multi-step form, - or a server-side cache keyed by user. If the client's second request lands on a different backend that has never seen them before, that state simply isn't there, and the user appears logged out or their cart is empty. ## How affinity is implemented The most common implementation is **cookie-based** and requires an L7 (application-layer) load balancer, because it depends on reading and writing HTTP headers. On a client's first request, the balancer picks a backend using its normal algorithm, then sets a cookie in the response - either an application-generated session cookie the balancer inspects, or a cookie the balancer itself injects (commonly named something like `AWSALBAPP` or similar depending on vendor) whose value encodes which backend handled the request. On every subsequent request, the balancer reads that cookie and routes directly to the identified backend, bypassing the normal algorithm entirely. A cruder alternative, usable at L4, is **IP-hash affinity**: the balancer hashes the client's source IP and consistently maps that hash to the same backend. This requires no application cooperation but is fragile in practice: - many clients share a source IP behind NAT or a corporate proxy, causing unwanted clustering onto one backend; - mobile clients that switch networks (WiFi to cellular) change IP mid-session and lose affinity anyway. ## The first trade-off: uneven load The first real trade-off is against even load distribution. A well-tuned algorithm like least-connections balances instantaneous load across the fleet, but sticky sessions override that per client: if, by chance, a disproportionate number of long-lived, chatty sessions get pinned to one backend early on, that backend stays hotter than its peers for as long as those sessions live, and the balancer cannot rebalance them away without breaking affinity. This is a slow-building form of load skew that doesn't show up in short benchmarks but becomes visible over the life of a long-running deployment with many concurrent user sessions. ## The second trade-off: the pinned backend goes away The second, more serious problem is what happens when the pinned backend actually goes away - a crash, an out-of-memory kill, a rolling deployment replacing it, or the health check pulling it for being unhealthy. Because the session's state lived only in that backend's process memory, that state is gone the instant the instance is. The load balancer's typical fallback is to re-pin the client to a different backend on their next request (since the original is no longer in the pool), but that new backend has no record of who this client is - the practical result depends on the application: - a poorly designed one shows the user as logged out or their cart empty; - a well-designed one degrades gracefully by re-authenticating from a token or re-fetching cart contents from a database. This is precisely the failure mode that makes stateful, affinity-dependent architectures operationally fragile: every backend restart, deploy, or crash becomes a mini data-loss event for whichever sessions happened to be pinned there, and it directly conflicts with elastic autoscaling, since scaling a stateful fleet down means deliberately evicting sessions. ## The fix: make the backends stateless The standard, and now near-universal, fix is to eliminate the need for affinity at the correctness layer by making backends **stateless**: session state is moved into a shared, external store that every backend instance can reach equally. - a Redis or Memcached cluster for session objects, - a database table for cart contents, - or, avoiding server-side storage entirely, a signed client-held token (a JWT) that carries the session's claims and is verified independently by whichever backend receives it. Once backends are stateless in this sense, session affinity becomes purely a performance optimization rather than a correctness requirement - it can still help by improving cache locality (a backend that has recently handled a user's requests may have warm in-memory caches for their data) without the risk that losing the pinned backend causes any actual data loss, since the shared store or token survives independently. This statelessness is also precisely what makes horizontal autoscaling and rolling deployments safe: any backend can be added, removed, or replaced at any time without special-casing which sessions were 'living' on it. Practically, teams should treat 'do we need sticky sessions for correctness, or only for locality' as a design question to answer explicitly, because defaulting to stateful sticky sessions without a shared backing store is a common source of subtle, hard-to-reproduce production bugs around deploys and autoscaling events.