Threaded vs Event-Loop Models

## The question behind the question Every server-side web framework has to answer one thing before it answers anything else: when a request arrives, **what unit of execution carries it from the first byte in to the last byte out**, and what happens to that unit while the request is waiting on something slow. Almost all the differences people attribute to "async frameworks" fall out of that single answer, so it is worth being able to describe the three common shapes precisely. ## Thread-per-request The server keeps a pool of worker threads. When a request is ready to be dispatched, one worker is borrowed, your handler runs **on that worker, top to bottom**, and the worker is returned once the response is written. - The code is ordinary sequential code: a call to a database or another service simply parks the worker until the answer comes back. - Everything the handler touches — the call stack, per-request values held against the running thread, the debugger's view — lines up with one request. - The waiting is the cost. A handler that spends 200 ms waiting on a downstream service holds a worker for 200 ms without using any processor time. Note the common misreading: "thread-per-request" almost never means a fresh operating-system thread is created per request. Threads are pooled and reused; creating one per request is what the pool exists to avoid. ## Event-loop A small number of loop threads — typically sized to the number of processor cores, not to the number of requests — sit in a readiness loop. Each turn of the loop picks up work that is ready (bytes arrived on a socket, a timer fired, a downstream response completed) and runs the corresponding piece of handler code **to completion of that turn**, then moves on. - A handler is therefore not one continuous run: it is a chain of short turns, stitched together by callbacks or completion values. - Nothing waits on a loop thread. The handler registers interest in a result and returns; the loop serves thousands of other sockets in the meantime. - Concurrency stops being bounded by thread count. Idle or slow connections cost memory and bookkeeping, not a thread each. ## Coroutine / suspending A coroutine framework runs handlers as resumable units of work on a small pool of carrier threads. The handler *looks* sequential — you write the downstream call as if you were waiting for it — but at each suspension point the runtime saves the handler's position, releases the carrier thread for other work, and resumes the handler when the result arrives. - You get thread economics close to the event-loop model with control flow close to the thread-per-request model. - The resumption may land on a **different** carrier thread, and that is the detail most people miss, because anything tied to the identity of the running thread stops being reliable across a suspension point. ## Side by side | | Thread-per-request | Event-loop | Coroutine | |---|---|---|---| | Runs the handler | one pooled worker, whole request | a shared loop thread, one turn at a time | a carrier thread, between suspensions | | While waiting on I/O | the worker is held, idle | nothing is held; the loop serves others | the carrier thread is released | | Handler code shape | plain sequential | callbacks / completions | sequential with suspension points | | Natural in-flight ceiling | the worker count | memory and file-handle limits | task count, not thread count | | Cost of a slow blocking call | one worker parked | one loop thread unavailable for every socket it owns | one carrier thread parked | ## What actually follows from the choice 1. **None of the models makes a single request faster.** They change how *cheaply the server can hold many requests at once*, which is a throughput and memory story, not a latency one. A handler that needs 200 ms of downstream time still takes 200 ms. 2. **The cost of blocking moves.** In the first model a blocked handler costs one worker out of many; in the other two it occupies a thread that was meant to serve a large share of the server's sockets, which is why those frameworks are strict about what may run inline. 3. **Thread identity stops being a reliable key.** Under the second and third models, one thread interleaves many requests and a single request may touch several threads, so per-request state must travel with the request rather than with the thread. 4. **The model is largely inherited, not chosen per endpoint.** A framework commits its whole surface — its client libraries, its middleware chain, its testing story — to one of these shapes, so this is a property of the framework you adopt rather than a switch you flip later. Being able to say which shape you are working in, and therefore what a wait costs, is the entry-level version of every capacity conversation that follows.