How would you choose a thread-per-request, event-loop or coroutine web framework for a new service, and what constrains that choice?

## Frame it as a commitment, not a preference A web framework's execution model is not a setting. It determines how handlers are written, which client libraries can be used unmodified, how errors and cancellation propagate, what a stack trace looks like, how tests are written and what your team can debug at three in the morning. Changing it later is close to a rewrite of every handler. So the decision deserves the same scrutiny as choosing a persistence technology, and the honest default is: **pick the simplest model the workload allows, and make the case in writing for anything more complex.** ## Start from the workload Ask what a request actually spends its time doing, and how many connections exist at once. | Workload characteristic | Pushes toward | Why | |---|---|---| | Many connections, mostly idle or long-lived | Loop or coroutine | Holding a thread per connection is the binding cost | | One request fans out to several slow dependencies | Loop or coroutine | Waiting dominates occupancy, and waiting is what these models make cheap | | Processor-bound work per request | Thread-per-request | No model removes computation; a core-sized pool is the honest ceiling | | Modest concurrency, latency dominated by one dependency | Thread-per-request | The bottleneck is downstream; the server model changes nothing | | Mixed, with a few long-lived endpoints | Either, with isolation | Separate capacity for the long-lived endpoints matters more than the model | The trap in this table is the second row read carelessly: a model that makes waiting cheap raises how many requests you can *hold*, not how fast any one completes, and holding more in flight simply pushes more concurrent load onto the dependency you were already waiting for. Without limits downstream, the faster server just moves the queue. ## Then check the ecosystem — this is usually decisive 1. **Are the clients and drivers you need non-blocking?** If the data store, message system or internal client libraries only offer blocking calls, a loop-based framework means every one of them must be pushed onto separate execution. You then pay the complexity of the asynchronous model and keep the thread costs of the blocking one. 2. **Does the surrounding tooling follow?** Tracing, metrics, request context, transaction handling and testing helpers all behave differently once a request spans threads. Maturity here is worth more than benchmark numbers. 3. **Does cancellation actually propagate?** A client disconnect should ideally stop downstream work; whether it does depends on the framework and the libraries, and it varies far more than headline throughput does. ## Then count the human costs - **Debuggability.** A sequential stack trace that shows the whole request is worth real money during an incident. Callback-stitched execution loses that unless the runtime reconstructs it; coroutine models usually keep much of it. - **Failure modes are harder, not easier.** Non-blocking servers accept work far past the point a thread pool would have pushed back, so overload arrives as memory growth and unbounded latency instead of a visible queue. You must add explicit admission control that the simpler model gave you for free. - **Hiring and handover.** The pool of people who can safely modify handler code differs between the models, and so does the number of ways a newcomer can accidentally block something shared. ## Recent shifts worth acknowledging Lightweight user-space threads have made blocking-style code cheap on several platforms: a handler that waits no longer pins a costly operating-system thread, which removes much of the historical reason to leave the sequential model. That does not make the loop model obsolete — a loop still gives tight control over scheduling and memory per connection — but it does mean "we need throughput, so it must be non-blocking" is no longer automatically true, and a principal-level answer should say so rather than reach for the newest model by reflex. ## How to actually decide 1. **Measure or estimate occupancy per request** and the expected concurrent connection count. These two numbers decide whether thread-holding is the constraint at all. 2. **Inventory the libraries** you must use and mark each blocking or non-blocking. If the important ones are blocking, the decision is largely made. 3. **Prototype the ugliest endpoint**, not the simplest: the one that streams, fans out, or needs cancellation. Frameworks look equivalent on a hello-world benchmark. 4. **Write down what you give up** — debuggability, library choice, the team's familiarity — and what observable metric would prove the decision wrong. 5. **Prefer isolating the exception.** If one workload genuinely needs different economics, giving it its own service is often cheaper than making every handler in the system pay for it. 6. **Decide how overload will be handled** in whichever model wins, because a concurrency limit and load shedding are required either way, and only the simplest model comes with a crude version of them built in. The answer an interviewer is listening for is not a favourite model. It is the recognition that the workload and the available libraries decide, that the model is stack-wide and hard to reverse, and that no model removes downstream latency.