skip to content

How does os.process_cpu_count() differ from os.cpu_count(), and what does it honour?

level: middleimportance: should knowfreq 35%

answer

  1. One counts the machine, one counts your permission
  2. Pinning narrows it; a quota does not
  3. Added alongside an interpreter-wide override
  4. Pool defaults were repointed at it in 3.13

basics

~10 s

os.cpu_count() reports the machine's logical processors; os.process_cpu_count(), added in Python 3.13, reports the ones the calling thread may actually run on, honouring the CPU affinity mask. Both are overridden by PYTHON_CPU_COUNT.

solid answer

~40 s

`os.cpu_count()` answers "how many logical CPUs does this machine have". `os.process_cpu_count()`, new in **3.13**, answers "how many may this process's calling thread use" — on Linux that means it respects the CPU affinity mask, so a process pinned to four processors on a 64-way host reports 4 rather than 64. Both return `None` if undeterminable, and both are overridden by the `-X cpu_count` interpreter option or the `PYTHON_CPU_COUNT` environment variable. **3.13** also repointed the default worker counts of `concurrent.futures.ProcessPoolExecutor`, `concurrent.futures.ThreadPoolExecutor` and `multiprocessing.Pool` at the process-aware function. Neither function sees a control-group CPU quota, which is a time budget rather than a set of processors.

code

console · 2 lines
console
python3 -X cpu_count=4 -c "import os; print(os.cpu_count(), os.process_cpu_count())"
PYTHON_CPU_COUNT=2 python3 -c "import os; print(os.cpu_count(), os.process_cpu_count())"

go deeper

for a junior

Remember that the os module offers two counts and that the newer one is the safer default for sizing concurrency. Knowing which name to reach for is enough at this stage.

for a middle

Explain the difference precisely: machine processors versus the processors this thread may be scheduled onto, the affinity mask that creates the gap, and the fact that neither sees a control-group time quota.

for a senior

Be ready to say when each is correct in production, why the standard-library pool defaults were repointed in 3.13, and how you would combine the affinity-aware count with a quota read from the control group.

for a principal

Decide where the number is set for the whole estate — an injected interpreter-wide override versus an explicit argument at every call site — and defend the loss of honesty that an override introduces for non-sizing callers.

### Two counts, two questions Python has offered `os.cpu_count()` for a long time, and it answers a hardware question: how many logical processors does this machine expose — cores multiplied by hardware threads. **Python 3.13** added `os.process_cpu_count()`, which answers a permission question: how many logical processors may the calling thread of this process actually be scheduled onto. Both return an `int` or `None` when the platform cannot determine the value. On a developer laptop the two agree, and that is why the distinction is easy to miss. They diverge the moment something restricts where the process may run. ### The affinity mask On Linux each thread carries a CPU affinity mask: the set of processors the scheduler is permitted to place it on. A batch scheduler, a pinning wrapper such as `taskset`, or an orchestrator that assigns exclusive cores will narrow that mask. Narrowing it genuinely removes processors from the process's reach, and `os.process_cpu_count()` reflects that — a process pinned to four of a host's sixty-four processors reports 4. `os.cpu_count()` still reports 64, because the machine still has 64. This is the case where the older function is not merely imprecise but actively wrong for sizing: the extra workers can never be scheduled anywhere useful, so they add memory and context switching for no parallelism at all. ### What neither function sees A control-group CPU quota is a different kind of restriction. It does not narrow the set of processors; it caps how much CPU *time* the group may consume per period. The group's threads still run on any processor, they are simply stopped once the allowance for the period is spent. Because no processor is removed from the process's reach, the affinity mask is untouched and `os.process_cpu_count()` reports the full host count exactly as `os.cpu_count()` does. So the useful mental model is: `os.process_cpu_count()` covers *pinning*, and nothing in the `os` module covers *quota*. The quota has to be read from the control group's own files — `cpu.max` on cgroup v2, which holds either a limit and a period or the word `max`, and the `cpu.cfs_quota_us` / `cpu.cfs_period_us` pair on v1 — and reconciled with the affinity-aware count by taking the smaller of the two. ### The override **3.13** also added a deliberate escape hatch: the `-X cpu_count` interpreter option and the equivalent `PYTHON_CPU_COUNT` environment variable. Setting either makes *both* functions return the supplied value for the life of the interpreter. This is the intended way for a deployment system to tell Python what it is really allowed to use, and it is far more robust than patching every call site, because every library that quietly sizes something from the standard functions picks up the corrected number too. The tradeoff is honesty: the override is process-wide and indiscriminate. Anything asking for the machine's size for a legitimately different reason — a diagnostic banner, a hardware inventory — also gets the faked value. In practice that is almost always the trade you want, but it should be a conscious one. ### Why the standard-library defaults moved Before **3.13**, `concurrent.futures.ProcessPoolExecutor()` with no `max_workers` sized itself from `os.cpu_count()`; from 3.13 it uses `os.process_cpu_count()`, as does `multiprocessing.Pool()` when given no process count, and `concurrent.futures.ThreadPoolExecutor()` in its `min(32, cpu + 4)` formula. That change means a pinned process on a modern interpreter gets a sane default for free, where on 3.12 it would have started a worker for every processor on the host. It does not rescue the quota case, which still needs an explicit `max_workers` or a `PYTHON_CPU_COUNT` injection. ### Choosing between them Use `os.cpu_count()` when you genuinely mean the hardware: logging what machine a job landed on, or reporting capacity. Use `os.process_cpu_count()` — with `or 1` — whenever the number is going to size concurrency, and then clamp it against the quota if one exists. A one-line helper that does that reconciliation, called once at startup and logged, saves an enormous amount of confused profiling later. One last habit worth forming: neither function is free of `None`, and both are cheap enough to call but not free, so read the value once at startup into a module-level constant rather than calling it inside a hot path or, worse, recomputing it per submitted work item. Capturing it once also gives you a single place to log it, which turns the question "how many workers did this process think it could run" from an archaeology exercise into a grep.

  • If a process is pinned to four processors, which of the two functions changes, and why?
    Only `os.process_cpu_count()`. Pinning narrows the thread's CPU affinity mask, which genuinely removes processors from the set the scheduler may use, and that function is defined in terms of that set. `os.cpu_count()` is defined in terms of the machine, so it keeps reporting every logical processor on the host regardless of where this particular process is allowed to run.
  • Why does os.process_cpu_count() still report the host count under a CPU quota?
    Because a quota restricts CPU *time*, not the set of processors. The group's threads remain eligible for every processor on the host; they are simply stopped once the group's allowance for the current period is spent. The affinity mask is untouched, so there is nothing for the function to reflect. The quota must be read from the control group's `cpu.max` file and reconciled by hand.
  • What is the downside of setting PYTHON_CPU_COUNT rather than passing max_workers explicitly?
    It is process-wide and indiscriminate: every caller of either function sees the substituted value, including code that wanted the machine's real size for diagnostics or capacity reporting. That breadth is also its strength — libraries that size something internally get the corrected number without any code change — but it makes the value a deployment-time fact that is invisible in the source, so it should be logged at startup.

saying these in an interview costs you the question

  • Says os.process_cpu_count() is quota-aware
  • Thinks the two functions are aliases for each other
  • Believes PYTHON_CPU_COUNT changes only one of the two
  • Assumes pool defaults have always used the process-aware count
  • Cannot say what an affinity mask restricts

context