Why can a ThreadPoolExecutor of 8 workers end up with hundreds of live threads in one process?

### Pools compose by multiplication Every sizing decision in a process is usually made in isolation: a helper module builds a pool sized to the machine because that is a sensible default for a helper module, and the caller builds a pool sized to the machine because that is a sensible default for a caller. Composed, the defaults multiply. Eight outer workers, each entering a function that creates a four-worker inner pool, means thirty-two worker threads plus the eight outer ones, doing eight cores' worth of useful work at best. The multiplication has three common sources. The first is an explicit inner `ThreadPoolExecutor` created inside the task function, often as a convenience for parallelising a small fan-out. The second is a compiled extension: a native numeric or compression runtime typically starts a thread team on first use, sized to the logical CPU count, and it does this once per *process*, invisibly, with no Python object you would notice. The third is the same helper called through a `ProcessPoolExecutor`, where each child interpreter independently repeats whatever the parent did — so a per-process inner pool becomes cores × cores threads across the machine. ### Why the extra threads cost more than they look Oversubscription is not free idling. Each OS thread reserves stack space, so a few hundred threads is a measurable and sometimes surprising memory line even when they are asleep. Runnable threads beyond the CPU count force the scheduler to time-slice, and every slice ends with cold caches for the thread that resumes; for memory-bound inner loops that alone can halve throughput. Threads doing pure-Python work also serialise on the GIL, so multiplying them adds contention on the one lock without adding parallelism. And in a nested arrangement the failure is self-reinforcing: inner pools created per call are also created and destroyed constantly, paying thread start-up over and over. What makes it a diagnosis question rather than a design question is that nothing in the code says 256. The outer pool says 16, the inner default says 16, and the product exists only at run time. `threading.active_count()` in a health endpoint or a log line at start-up is the cheapest way to notice, and it is worth having before you need it. ### Fixing it The rule is to budget the product, not each pool. Decide how many runnable threads the process should have in total — for CPU-bound work, roughly `os.process_cpu_count()` — and then divide it across the nesting levels rather than letting each level claim it. In practice that means one of three shapes. Pin the inner level to one. A native runtime that reads a thread-count environment variable will honour a value of 1 if it is set **before the extension is imported**, because the team is usually sized once at load or first use; setting it afterwards from Python has no effect. This is the standard arrangement when the outer pool is the one you want to be wide: many single-threaded inner computations, parallelism owned at the top. Or pin the outer level to one and let the inner runtime spread. That is the right answer when the inner work is compiled, releases the GIL and parallelises well internally: one caller, one wide native thread team, no Python-level contention at all. Or keep both, sized deliberately, when the levels genuinely serve different purposes — an outer pool of a few threads for blocking I/O, an inner pool for compute — and check that the product is still the number you intended. Alongside the sizing, hoist pools out of task functions. An executor created inside a function that runs thousands of times is both a multiplication and a churn problem; one module-level executor, or an executor passed in as an argument, fixes both and makes the total count auditable by reading the code. If a library you call insists on managing its own pool and gives you no way to size it, that is a genuine reason to run it in a process pool with a single-threaded inner configuration rather than inside your thread pool. ### The interview shape This is a differentiator rather than a screening question: many strong Python engineers have never hit it, because it only bites when compute-heavy libraries meet a hand-rolled pool. But it is the concrete meaning of the word oversubscription, and the answer an interviewer is listening for is the multiplication itself — that concurrency levels compose by product, and that a sizing policy has to be stated for the whole process rather than pool by pool.