How would you choose a worker model for a new Python service, and what actually forces the decision?

## Start by measuring the handler, not by ranking the models Three numbers decide most of it. 1. First, the **split between waiting and computing** in a typical request: profile it, because intuition is wrong here surprisingly often, and a handler that looks like pure I/O frequently spends a third of its time serialising and validating. 2. Second, the **peak number of *simultaneous* connections** — not requests per second, which is a different quantity: a service with modest throughput but tens of thousands of long-lived idle sockets has a completely different answer from a high-throughput service with short requests. 3. Third, the **memory budget per instance**, since that sets how many processes you can afford before you are buying capacity rather than choosing an architecture. ## Then admit what actually forces the decision: the drivers The theory says async workers hold the most concurrency per byte. Reality says a model is only usable if every library on the request path cooperates. One synchronous datastore driver, one blocking client for an internal service, one file-format library with a synchronous read — and the async model degrades into an event loop that offloads to a thread pool, which is the threaded model with extra constraints and a new failure mode when the pool saturates. So the first question is not "which model is best" but "which models can my dependency set actually support today". Frequently only one can, and the design is settled before any tradeoff is weighed. ## Blast radius is the second forcing function Multiple processes are the only structure here with a **hard boundary**: a native crash, an unbounded allocation, or a wedged handler takes one worker and the supervisor replaces it. For services that decode untrusted input, or that load native extensions of uncertain quality, that boundary is worth paying several times the memory for — and it is the reason almost every production Python deployment has *some* pre-forking in it, whatever runs inside each worker. The corollary is that the models are not alternatives to be selected among; the real decision is **what to nest inside a small number of processes**. ## Weigh the costs that do not appear in a benchmark - An **async** codebase is a commitment that spreads: async colours call chains, constrains which libraries may ever be added, and demands a discipline every future contributor must hold. - **Threads** demand a different discipline — shared mutable state, per-request context, and lock ordering — which is more familiar but easier to get subtly wrong. - **Processes** demand the least discipline and the most memory, and they push state you might have kept in a local cache into a shared store instead. Also count the cost of *changing* the choice: moving a mature service from threaded to async is a rewrite of every handler and every dependency decision, so this is a decision to make with the ten-year version of the service in mind, not the prototype. ## How I would decide, concretely - If the handlers wait and the drivers are ordinary blocking libraries: pre-forked workers with a **thread pool** inside each, sized against downstream connection limits. - If the service holds very many mostly-idle connections and the whole stack is non-blocking: pre-forked workers with an **event loop** inside each, plus a loop-lag metric from day one so the first blocking call is caught in a deploy rather than an incident. - If handlers do real computation: keep it **off the request path** entirely — a queue and separate consumers — rather than trying to make any worker model absorb it. Mixed workloads should be split into separate deployments before they are made to share a model, because a service that must be both is a service where every model is the wrong one. ## What is genuinely new in 3.14, and how much to bet on it **Free-threading** is officially supported (PEP 779), which for the first time makes threads inside one process a real answer to CPU concurrency — at roughly 5–10% single-threaded cost and conditional on every native extension supporting it. **Multiple interpreters** reached the stdlib (PEP 734) with `concurrent.interpreters` and an `InterpreterPoolExecutor`, offering isolation closer to a process at a lighter footprint. Both are real and both are young. The principled position is to design so that either could be adopted — keep handlers free of process-global mutable state, keep the worker model a deployment concern rather than a shape baked into every module — and to run one of them behind a flag on a non-critical service before making a fleet-wide bet.