skip to content

Why does CPython raise a DeprecationWarning for os.fork() in a multi-threaded process?

level: middleimportance: must knowfreq 42%

answer

  1. Conditional on the process, not the call
  2. Only async-signal-safe work is legal after
  3. Inherited locks with no owner left
  4. Added in 3.12, followed by a default change
  5. forkserver on Linux, spawn on macOS in 3.14

basics

~20 s

Because the child gets the parent's whole memory but only the forking thread, so any lock the vanished threads held stays locked forever and the child deadlocks. Python 3.12 added the warning; Python 3.14 moved multiprocessing's Unix default to forkserver.

solid answer

~50 s

POSIX only guarantees that async-signal-safe operations are legal in the child of a fork from a multi-threaded process, and a Python child does far more than that. The warning, added in **Python 3.12**, fires from `os.fork()` and `os.forkpty()` only when the process has more than one running thread, and it flags a real hazard: internal locks — a `logging.Handler`'s, the import machinery's, a buffered writer's, the allocator's, a connection pool's — can be copied in the locked state, and the thread that would have released them does not exist in the child. The child then hangs the first time it touches that subsystem, often before any of your code runs. The warning is half a migration: on **Python 3.14** `multiprocessing` and `concurrent.futures.ProcessPoolExecutor` default to `forkserver` on Linux and other Unix, macOS and Windows stay on `spawn`, and plain `fork` must be asked for explicitly.

code

console · 8 lines
console
python3.14 -X dev -c '
import os, threading, time
threading.Thread(target=time.sleep, args=(2,), daemon=True).start()
pid = os.fork()
if pid == 0:
    os._exit(0)
os.waitpid(pid, 0)
'

go deeper

for a junior

Know what the warning means when you see it: this process has more than one thread and is about to fork, which is the unsafe combination. Do not silence it.

for a middle

Explain the mechanism precisely - one thread of control survives, all memory is copied, so locks held by other threads arrive locked with no owner - and name the release that added the warning and the release that changed the default.

for a senior

Demonstrate that you can find where the fork happens in a real service, decide between forking earlier and changing the start method, and weigh the loss of inherited state that comes with not forking the parent.

for a principal

Frame the tradeoff for the organisation: worker startup cost and inherited state versus a class of load-dependent deadlocks, plus the migration burden of code that quietly relied on fork inheriting live objects.

### What the warning is Since **Python 3.12**, calling `os.fork()` or `os.forkpty()` from a process that has more than one running thread raises a `DeprecationWarning`. Nothing else about the call changes: the fork still happens, the child still runs. The warning is a signal that this specific *combination* — forking plus threads — is a bug pattern the language is steering people away from, and it is the visible front of a multi-release migration whose other half landed in 3.14. Note what is **not** deprecated. `os.fork()` in a single-threaded process is an ordinary, supported Unix operation and emits nothing. The warning is conditional on the state of the process at the moment of the call. ### Why the combination is unsafe `fork()` copies the address space but creates the child with only one thread of control: the caller. Every other thread ceases to exist in the child, while all the memory those threads were working on arrives intact and mid-flight. The dangerous case is a lock. A lock that some other thread held at the instant of the fork is copied in the **held** state, and the thread that would have released it is not in the child. Nothing will ever release it, so the child's first attempt to acquire that lock blocks forever. There is no exception, no traceback, no exit code — just a process that stops. You rarely hit this through a lock you wrote. The locks that bite are the ones inside machinery you did not think about: - a `logging.Handler`'s lock, taken around every emitted record; - the import system's per-module locks, taken by any lazy `import` in the child; - a buffered writer's internal lock; - the memory allocator's own lock, which means even ordinary object creation can hang; - a connection pool's or a cache's internal structures. Which of those is locked at fork time is decided by the OS scheduler, so the failure is probabilistic and grows more likely the busier the parent's threads are — which is exactly why it tends to appear first in production and not in a test. POSIX has always said this outright: in the child of a fork from a multi-threaded process, only async-signal-safe functions may be called before `exec`. Python cannot honour that restriction — allocating an object takes a lock — so the only genuinely safe post-fork paths are "immediately exec" or "do nothing that touches inherited state". ### What Python 3.14 changed On **Python 3.14** the `multiprocessing` default start method is `forkserver` on Linux and other Unix platforms; **macOS and Windows default to `spawn`** (macOS has done so since 3.8), and `fork` must now be requested explicitly with `multiprocessing.set_start_method` or `multiprocessing.get_context`. `concurrent.futures.ProcessPoolExecutor` follows the same default because it uses `multiprocessing`'s context. `forkserver` addresses the thread problem structurally rather than by warning about it: a small helper process is started early, and every worker is forked from **that** process, which is single-threaded and does almost nothing. Your threaded parent is never the thing being forked. The price is that children no longer inherit state created after the helper started, so anything a worker needs must be picklable or re-created — a real cost, and the reason the change waited for a major release. (Choosing between the start methods on their broader merits belongs to the concurrency material; here the point is only that the default moved *because of threads*.) ### Seeing the warning at all `DeprecationWarning` is ignored by default outside `__main__`, so a service can be forking unsafely for years in silence. Make it visible deliberately: run with `-X dev`, or set `PYTHONDEVMODE=1` or `PYTHONWARNINGS=default::DeprecationWarning` in the environment, or call `warnings.simplefilter("default", DeprecationWarning)` during startup — and call `logging.captureWarnings(True)` so the message lands in the logs you actually read rather than on a stderr nobody collects. ### The response an interviewer wants Not "silence the warning". The warning is telling you that the process forks after it has grown threads, and the fixes are structural: 1. Fork before starting threads — build the process pool during startup, ahead of metrics, health-check and logging threads. 2. Or don't fork your process at all: take the 3.14 default of `forkserver`, or `spawn`. 3. Or, if the child immediately calls one of the `exec` family (`os.execv` and friends), the child's inherited locks never matter, because the whole address space is replaced. Suppressing the warning leaves you with a deadlock that appears under load, in a child, with no stack — one of the more expensive bugs to chase.

  • How would you see this warning in a service where warnings are normally silent?
    `DeprecationWarning` is ignored by default outside `__main__`, so make it explicit: run with `-X dev`, or set `PYTHONDEVMODE=1` or `PYTHONWARNINGS=default::DeprecationWarning` in the environment, or call `warnings.simplefilter("default", DeprecationWarning)` early in startup. Then call `logging.captureWarnings(True)` so the message reaches the logs you actually collect instead of a stderr stream nobody reads.
  • Does switching to forkserver remove the hazard completely?
    It removes it from the worker-creation path. The helper process is started early, while the parent is still single-threaded, and every worker is forked from that helper rather than from your threaded process. It does not protect an `os.fork()` you call yourself, and it costs inherited state: children see nothing created after the helper started, so what workers need must be picklable or rebuilt in the child.
  • Why is a direct exec after the fork considered safe even from a threaded process?
    Because `exec` replaces the entire address space. Inherited locks, half-finished buffers and orphaned data structures are discarded along with the old image, so nothing in the child ever tries to acquire them. That is the classic fork-then-exec pattern, and it is why `subprocess` is unaffected by this hazard while a fork that keeps running Python code is not.

saying these in an interview costs you the question

  • Says os.fork() itself is deprecated in Python 3.14
  • Claims the warning is about copy-on-write memory cost
  • Thinks the child resumes the parent's other threads
  • Calls forkserver simply a faster kind of fork
  • Proposes suppressing the warning as the fix
  • States that fork is still the Linux default in 3.14

context