Pool Sizing and Oversubscription

### Two different questions `os.cpu_count()` and `os.process_cpu_count()` look interchangeable and are not. The first is an inventory question — *how many logical CPUs does this machine have?* — and it answers with the whole box, including CPUs this process will never be scheduled onto. The second, added in **Python 3.13**, is a permission question: *how many logical CPUs may the calling thread actually run on?* On Linux that is the size of the thread's CPU affinity mask, which a launcher, a batch scheduler or an orchestration layer may have narrowed. On a 64-CPU host where the process has been pinned to four of them, `os.cpu_count()` says 64 and `os.process_cpu_count()` says 4. Both can return `None`: the count is genuinely undeterminable on some platforms, and code that writes `os.process_cpu_count() + 4` will then raise `TypeError` at start-up on exactly the machine you cannot debug. The idiom is `(os.process_cpu_count() or 1)`. ### Why the difference bites when sizing a pool For CPU-bound work the whole point of a pool is to keep every core you own busy and no more. Threads or processes beyond that count do not add throughput — they add context switches, cache pressure, and for processes, memory. Sizing 64 workers onto a 4-CPU allowance means each unit of work is interleaved sixteen ways: total completion time is unchanged at best, and worse once the scheduler and the caches start paying for the churn. Memory is the harsher penalty for a process pool, where every worker is a full interpreter. The standard library moved with this in 3.13. `ProcessPoolExecutor` and `multiprocessing.Pool` default their worker count to `os.process_cpu_count()`; before 3.13 they used `os.cpu_count()`. `ThreadPoolExecutor`'s default became `min(32, (os.process_cpu_count() or 1) + 4)` — the `+ 4` is a small allowance for I/O-bound tasks that spend most of their time blocked, and the cap of 32 stops a very large machine from spawning an absurd number of threads for what is usually I/O work. On Windows a process pool is additionally capped at 61 workers by an operating-system wait limit. So on 3.14, code that simply omits the worker count already gets the better number. Code that hardcodes `ThreadPoolExecutor(max_workers=os.cpu_count() * 2)` — a very common line — does not. ### What the functions do *not* tell you Affinity is a *set* restriction: which CPUs you may run on. A quota is a *rate* restriction: how much CPU time you may consume per period. Only the first narrows `os.process_cpu_count()`. A process allowed a half CPU of time but permitted to run anywhere still sees the full machine count, sizes a pool for it, saturates every core in bursts and then gets throttled hard. This is the single most common surprise in constrained deployments, and it is why the number sometimes has to be told rather than detected. Python 3.13 added exactly that knob: `-X cpu_count=n` on the command line, or the `PYTHON_CPU_COUNT` environment variable. Either one overrides what **both** functions report for the whole process, so every library that sizes itself from them follows without being patched individually. Passing the literal value `default` restores normal detection. Note what the override does not do: it changes what CPython *reports*, not what the operating system permits — it is a sizing hint, not a sandbox. ### Physical versus logical Neither function distinguishes physical cores from hardware threads; both count logical CPUs. For workloads whose per-core work is memory-bound or already vector-saturated, two hardware threads on one physical core do not give two cores' throughput, and a pool sized to the logical count can measure slower than one sized to the physical count. Python does not expose a physical-core count in the standard library, so if that distinction matters for your workload, it belongs in configuration measured on the target machine rather than in a detection call. ### The practical rule Size CPU-bound pools from `(os.process_cpu_count() or 1)`, never from `os.cpu_count()`. Let the executor defaults do it when you have no better information, since since 3.13 they already use the right function. Allow an explicit override in configuration for every pool, because detection cannot see a quota. And treat `os.cpu_count()` as what it is: a fact about the hardware, useful for logging and diagnostics, and almost never the right input to a sizing decision.