Start Methods: spawn, fork, forkserver

A **start method** is the recipe `multiprocessing` uses to bring a worker process into existence. CPython 3.14 ships three: `fork`, `spawn` and `forkserver`. The `__main__` guard question is really a question about what `spawn` has to do that `fork` does not. ### What spawn actually does When you call `Process.start()` under `spawn`, the parent launches a *new* Python interpreter (roughly `sys.executable` with a bootstrap argument), hands it a pipe, and sends it a small pickled payload: the target callable, the arguments, and some process state. Nothing of the parent's memory is inherited — no imported modules, no open objects, no module-level variables. That creates a bootstrapping problem. A function is not pickled by value; it is pickled **by reference**, as "the name `work` in module `__main__`". For the child to unpickle that reference it must first *import* the module the function came from. So the child imports your script — under the name `__mp_main__`, precisely so `__name__ == "__main__"` is false there — and only then resolves the target and runs it. ### Why the guard is mandatory Importing your script executes every top-level statement in it a second time, in every child. If a top-level statement starts processes, each child would start more children, which would import the module again, and so on. Rather than let you fork-bomb your own machine, CPython checks during child bootstrap whether a process is being started before the current process has finished bootstrapping, and raises `RuntimeError` with a message that spells out the fix: put the process-starting code inside `if __name__ == "__main__":`, and call `multiprocessing.freeze_support()` if the program is being frozen into an executable. The rule of thumb is therefore: **definitions above the guard, actions below it.** Imports, function and class definitions, and constants must stay at module level, because the child needs them to resolve the target. Anything with a side effect — starting processes or pools, parsing `sys.argv`, opening files or network connections, writing logs, running a long computation — belongs under the guard, or it will run once per worker. ### The same rule for forkserver, not for fork `forkserver` also imports the main module in the worker, so it needs the guard too. `fork` does not: the child is a copy of the parent's address space and simply continues after the fork point, so nothing is re-imported and no guard is required. That is exactly why so much older code omits the guard and still worked on Linux — and why it started failing on Python 3.14, where the default on Unix other than macOS became `forkserver`. macOS has defaulted to `spawn` since 3.8, and Windows has only ever supported `spawn`. ### Related consequences of the re-import Because the child re-imports rather than inherits, module-level mutable state that the parent filled in *after* import does not travel. A cache populated inside `main()` is empty in a `spawn` or `forkserver` child, while under `fork` it would arrive pre-filled. Code that quietly depended on that inheritance breaks when the start method changes, and the failure looks like a logic bug rather than a concurrency bug. The re-import also means the main module must be **importable at all**. Interactive sessions have no importable `__main__`, which is why `spawn` workers defined in a REPL cannot resolve their target; the target function must live in a real, importable module. ### What the error looks like A missing guard under `spawn` surfaces as a `RuntimeError` raised in the child during bootstrap, usually repeated for each worker and often accompanied by a partially started pool. It is not a hang and not a pickling error — recognising the bootstrap message is the fastest route from symptom to fix. If instead you see the *parent's* top-level work being redone with no error, you are usually looking at side-effecting module-level code that is legal but wasteful: it runs once per worker, multiplying startup cost by the pool size. ### The cost that follows from the same fact Because every worker imports your module graph, a spawned worker's start-up cost scales with how heavy your imports are rather than with how much work the task does. A pool created once at start-up pays that cost once per worker; a pool created inside a request handler pays it on every request, which is how a change of start method turns a fast endpoint into a slow one without a single line of the task changing. The same reasoning explains why an expensive table or model loaded at module level is a poor fit for `spawn`: it is rebuilt in each worker instead of being inherited. Pass it as an argument, load it once in a pool initializer, or keep the pool alive across requests.