A service stores per-request data such as the caller's identity in thread-local storage while handling requests on a pool of reusable worker threads. What can go wrong across requests, and how do you prevent it?

## Why pooling turns a convenience into a hazard Thread-local storage keys state to the thread. That is fine when the thread's life and the work's life coincide — a thread created for one task, ending with it, taking its slots with it. Pools break that coincidence deliberately: a worker exists for the life of the process and executes an unbounded sequence of unrelated tasks. Its per-thread map persists across all of them. So every entry a task writes is, by default, part of the *next* task's initial state. Nothing clears it. The pool does not know your keys, and there is no per-task lifecycle to hook into unless you add one. ## Failure one: unbounded memory growth Each worker holds one entry per key ever set on it, and each entry pins its value — which may be a small string or may be a request object holding a payload, a connection, a whole parsed document, or a reference chain into a large cache. Multiply by workers. The result is heap growth that correlates with pool size and time rather than with concurrent load, so it looks like a slow leak with no obvious owner: heap dumps show live objects retained by *thread* references, not by anything in your request path. A related, nastier variant: an entry pins a value whose class was loaded by a module or plugin classloader. As long as the worker lives, that classloader cannot be collected, so an entire module's classes stay resident after unload/reload. That is the classic redeploy leak, and its root cause is exactly this lifetime mismatch. ## Failure two: cross-request data bleed (the serious one) The correctness failure matters more than the memory one. Consider the sequence: ``` worker-3: request A (authenticated as alice) -> set USER = alice ; ... ; return [no clear] worker-3: request B (health check, no auth) -> code reads USER -> gets alice ``` Request B never authenticated, but a read of the ambient identity returns a real, valid-looking user. Depending on what consumes it, the consequences range from mislabeled logs and metrics attributed to the wrong tenant, through audit records naming the wrong actor, to authorization decisions and data queries executed as someone else. The last is a genuine data-disclosure vulnerability. What makes it brutal to find: - **It is scheduler-dependent.** It only manifests when a request lands on a worker whose previous task left a value, so it reproduces at low rates and never on an idle test machine. - **It looks legitimate.** The value is well-formed, present, and of the right type — nothing validates that it belongs to *this* request. - **The setter and the victim are far apart.** The code that forgot to clear is in one module; the damage appears in another that merely reads ambient context. The same shape applies to every ambient value: tenant, locale, feature-flag overrides, trace/span ids (which corrupt traces by attaching new work to a finished span), transaction or database-session handles (which can join work to a stale transaction), and security-role sets. ## Prevention: pair every set with a removal The rule is mechanical: **never call set without a paired remove in a finally block**, and never leave it to individual handlers to remember. ``` withContext(ctx, body): CONTEXT.set(ctx) try: body() finally: CONTEXT.remove() // remove, not set(null) ``` Two details matter. **Remove, do not null out.** Setting a null/empty value still leaves an entry in the map, so it still consumes a slot and, in some implementations, still participates in the structures that keep the map alive. Removal is what releases the reference. Nulling also makes "present but empty" indistinguishable from "deliberately cleared" in diagnostics. **Apply it at the boundary.** Wrap the point where a task begins — the server's request filter, the pool's task decorator, the message consumer's loop — not in each handler. A convention that every handler must remember is a convention that will be violated, and the violation is silent. One decorator around task execution guarantees the invariant regardless of what the handler does, including when it throws. ## Defence in depth **Fail closed on missing context.** If code needs an identity and the slot is empty, that must be a loud error, not a fallback. This flips the failure from silent-and-wrong to obvious-and-safe, and it removes the incentive to "leave the last value around just in case". **Stamp and validate.** Store a task/request identifier alongside the context and have readers assert it matches the task they are running. A mismatch means stale context and should throw. Cheap, and it converts an invisible bleed into an immediate exception naming both requests. **Assert cleanliness at boundaries in tests.** After each task in an integration test, assert the worker's known slots are empty. This catches the missing-finally the moment it is introduced rather than months later in production. **Keep values small and immutable.** A leaked immutable string is a bounded, harmless leak; a leaked mutable request object retains a graph and can be mutated by whoever finds it. **Prefer scoped bindings where the platform offers them.** Mechanisms that bind a value only for the dynamic extent of a call and unbind automatically on exit make the leak structurally impossible instead of relying on discipline. Where such a mechanism exists, it is strictly better than manual set/remove. ## Diagnosing an existing case If you suspect bleed: take a heap dump and look for objects retained via thread references from pool workers — that names both the leaking key and its value type. For the correctness symptom, correlate logs where the identity or tenant on a request disagrees with its authenticated principal, and check whether affected requests share a worker thread name with an earlier request from the other party. Thread names are the thread you pull.