You can serve each request either on its own operating-system thread or on its own lightweight, runtime-scheduled thread over a small host pool. How do you decide, and which assumptions elsewhere in the system does the lightweight model invalidate?

## Start with the arithmetic, not the model The honest form of this decision is a capacity calculation, and Little's law gives it: the number of requests in flight equals arrival rate multiplied by the time each one spends in the system. Ten thousand requests per second with a 100 ms residence time is a thousand in flight; the same rate at 2 s residence — a slow third party, a long poll, a streaming connection — is twenty thousand. Thread-per-request means in-flight count *is* thread count. So the comparison is: - **Is that count affordable as OS threads?** Each carries a contiguous stack reservation, a non-pageable kernel stack, and scheduler presence. A few thousand is routine; tens of thousands is strained; hundreds of thousands is not a design. - **Is the work waiting or computing?** Compute throughput is bounded by cores under either model. If requests are CPU-bound, lightweight threads change memory and creation cost only — and can slightly hurt, since the runtime scheduler lacks the kernel's view of the machine. If requests are mostly waiting on I/O, the thread is idle inventory, and making that inventory cost kilobytes instead of megabytes is precisely the win. - **How long-lived is a request?** Long-lived, mostly-idle connections are the strongest case for lightweight threads; short CPU-heavy ones the weakest. If the numbers say a bounded pool of a few hundred OS threads carries the load with acceptable latency, the simpler model is the right answer, and the case for change has not been made. ## The precondition that decides feasibility The lightweight model is a bargain: the runtime mediates every wait, so a blocked task costs a heap continuation rather than a host thread. That bargain holds only for waits the runtime implements. Audit the hot path for blocking it cannot mediate — native drivers, synchronous file access, platform name resolution, third-party libraries with their own thread and locking discipline. Each such call pins a host for its duration, collapsing effective parallelism to the unpinned hosts and, in the limit, deadlocking or forcing the runtime to compensate by spawning hosts until the memory profile is back where it started. This audit is a go/no-go gate, not a tuning task. ## What the change invalidates **Backpressure.** This is the most consequential and the most often missed. A pool of 200 threads was never only an execution mechanism; it was an admission-control limit. Requests beyond it queued or were rejected, which protected the database connection pool, downstream services and memory. Give every request its own thread and that ceiling disappears: load passes straight through to whatever is actually scarce, and the failure mode moves from "requests queue at the front door" to "the database is at 100% and everything times out". Replace it deliberately — bounded queues at the edge, semaphores in front of each scarce resource, and concurrency limits per dependency. **Thread identity.** Code that pools or caches by thread — an object per thread, a connection bound to a thread, a buffer reused across requests on the same thread, a lock whose ownership is recorded against the host — assumes threads are few and long-lived. With a thread per request, per-thread caches stop being caches (each is used once) and become allocation storms, while anything keyed to *host* identity is now plainly wrong because the host changes across a park. **Per-thread memory.** A 64 KB I/O buffer per thread is 13 MB at 200 threads and 13 GB at 200,000. Any per-thread allocation must be re-sized for the new order of magnitude or moved to a pooled resource. **Operating-system controls.** Priorities, scheduling classes, CPU affinity, per-thread CPU accounting and resource limits attach to hosts, not to lightweight threads. If you were using thread priority to protect latency-critical work, that lever is gone and must be reimplemented in the application — separate host pools, explicit queues with priority, or admission classes. **Observability and operations.** Thread dumps, per-thread metrics, profilers and tracing were built for hundreds of threads. A million lightweight threads breaks list-oriented tooling; per-thread metric series explode cardinality; a stack trace may show the dispatcher rather than the logical task unless the runtime renders it. Verify that the runtime's diagnostics answer "what are my tasks doing?" before you depend on the model in production. **Cancellation and timeouts.** Cheap threads encourage fan-out, and fan-out without disciplined cancellation leaks work: the caller times out while ten spawned tasks keep running and keep holding downstream resources. Scoped lifetimes and propagated cancellation become mandatory rather than nice to have. ## Deciding and de-risking A defensible position sounds like this: *state the in-flight number and where it came from; state the waiting fraction; show that OS threads either do or do not fit that number at acceptable memory and latency; report the blocking-surface audit; and list the compensating controls you will add.* Then migrate at the edges — one endpoint or one service — measure throughput, latency percentiles, host-pool occupancy and pinning events under real load, and keep the explicit concurrency limits regardless of which model wins, because they are the part that protects the rest of the system either way. The answer that fails is "lightweight threads are modern, so we use them". The model is a representation choice for waiting work; it changes what concurrency costs, not what the machine can compute, and it removes safety properties that were being provided accidentally.