Why can multiprocessing's fork start method deadlock a child of a threaded service?

This is the failure that drove Python's whole retreat from `fork`, and it is worth being able to explain from first principles. ### The mechanism A fork produces a child whose address space is a copy of the parent's, but whose thread table contains exactly one thread: the one that called fork. Every other thread is gone — not stopped, not joined, simply absent. A lock, however, is not a thread. It is a small piece of state in memory saying "held" or "free", plus a wait queue. It gets copied like everything else. So if a background thread happened to hold a lock at the instant of the fork, the child inherits a lock in the held state whose owner does not exist and never will. The first time code in the child tries to acquire it, it blocks forever. Nothing raises; nothing times out. The interval during which this can happen is not a rare window. Locks are everywhere in a real process: the lock inside a logging handler, the internal locks of a C allocator, the mutex protecting a connection pool in a database or HTTP client library, buffer locks inside an I/O layer. A single background heartbeat, metrics or log-flush thread is enough to make the hazard permanent, because the moment of forking is uncorrelated with what that thread is doing. ### What it looks like in production Take a route-optimisation service that runs a background thread emitting progress logs while the request handler forks a pool of workers to score candidate routes. It works for months, then a change to logging or a library upgrade widens the window a little and workers begin to hang — not always, not reproducibly, and typically under load, when the background thread is busiest. The pool's `join()` never returns, the queue never drains, `Process.exitcode` stays `None`, and there is no traceback because the child is not crashing, it is waiting. Chasing it across a three-week release train is miserable precisely because the trigger is timing, not code path. When you suspect it, the useful moves are: check whether the parent is multi-threaded at the moment of the fork; take a stack of the stuck child from outside the process (CPython 3.14's remote debugging support via `sys.remote_exec()`, or a native debugger) and look for a thread parked in a lock acquire during library initialisation; and confirm that the same workload with `spawn` does not hang. A child stuck before it prints anything at all, on its very first log line or first client call, is the signature. ### Why fork is retreating rather than being fixed POSIX only guarantees that a child of a multi-threaded fork may safely call async-signal-safe functions until it execs. Python code is nowhere near that restriction. There is no way for the interpreter to know which library locks exist or how to reinitialise them, so the language has moved the default instead: macOS switched to `spawn` in 3.8 because forking after touching higher-level system frameworks crashes outright; Python 3.12 added a `DeprecationWarning` when the implicit default `fork` was used from a multi-threaded parent; and 3.14 changed the default on Unix other than macOS to `forkserver`. `forkserver` is the structural fix — the process that does the forking is a small, single-threaded server created before your threads exist, so no application thread can ever be holding a lock at fork time. ### The mitigations, ranked First, change the start method: `forkserver` keeps fork's cheap worker creation, `spawn` is the portable choice, and on 3.14 you are already on one of them unless you asked for `fork`. Second, if you must fork, do it *before* starting any threads — create the pool at start-up, keep it, and never create one from a request handler in a threaded server. Third, `os.register_at_fork()` lets you register callbacks that run before the fork, in the parent after it, and in the child after it, which is how you take your own locks around the fork and reinitialise your own state; libraries can use it for the state they own, but you cannot register on behalf of code you did not write. Fourth, re-establish anything connection-shaped in a pool initializer rather than inheriting it — an inherited socket or client handle is a second, independent hazard on the same fork. What does **not** work: adding a timeout and retrying (the lock never becomes free), catching an exception (there is none), or reducing thread count until the race "goes away" (it narrows the window and hides the bug).