A forked child reruns the parent's atexit cleanup in a 6-hour nightly reconciliation job, causing intermittent timeouts -- how do you find and fix it?

### Why an inherited registry is a production hazard `atexit.register(func)` appends to a list held in the interpreter. `os.fork()` copies the interpreter's memory, so the child starts life with an identical list. Nothing marks those entries as belonging to the parent, because at registration time there was only one process. When the child later shuts down normally, it runs them in last-in-first-out order exactly as the parent eventually will. The callbacks that get registered in a long-running batch job are precisely the dangerous ones: release a database advisory lock, delete a run-lock or pid file, return a pooled connection, mark a lease free in a coordination store, write a "finished cleanly" marker. Every one of those is a claim of ownership over something exactly one process owns. Run it twice and the second run is a lie. ### Why it presents as an intermittent timeout A nightly reconciliation job that forks a child per batch and runs for six hours has a race with a wide window and a narrow trigger. The child's early release of the lease is only visible when something else tries to acquire it before the parent finishes -- the next batch, a retry, a monitoring probe. Most nights the ordering is harmless; on the nights it is not, some later step blocks on a resource that the coordination store now believes is free (or, worse, has handed to another waiter) and eventually gives up on its acquire timeout. The timeout is reported far from the fork, in a step that has no obvious relationship to it, which is what makes the failure look flaky rather than causal. The tell that separates this from an ordinary lock-contention problem is *count*: the cleanup was performed more times than the job started units of work. If the resource was acquired once and released twice, contention is not the explanation. ### Confirming it The cheapest confirmation is to make the callbacks self-identifying. Print or log `os.getpid()` from inside every cleanup callback, and record the main process's pid once at startup. If a cleanup line appears under a pid that is not the main one, an inherited callback fired in a child and the diagnosis is done. A second confirming signal is ordering: the duplicate cleanup is logged long before the job ends, at the moment a batch child finished, not at shutdown. From there, read the fork site and answer two questions. First, how does the child leave -- does the branch end in `os._exit()`, or does it `sys.exit()`, `return`, or simply fall off the end? Second, is there a broad `except` between the fork and the exit that could swallow a `SystemExit` and let the child continue as a second copy of the program? Those two answers usually explain everything. ### The fix, in layers **Layer one: the child must hard-exit.** Flush anything the child wrote, then `os._exit(0)`, and wrap the child's body so that an escaping exception ends in `os._exit(1)` instead of in the normal exception path. This is the actual fix; the child now cannot reach interpreter shutdown, so no inherited callback can fire. **Layer two: make the callbacks refuse to run in the wrong process.** Record the owning pid when you register, and have the callback compare it with `os.getpid()` and return if they differ. This is a few lines and it converts a class of future bugs -- someone adds a fork somewhere else, a library forks under you -- from silent corruption into a no-op. It is the layer worth arguing for in review, because it survives changes to code you do not own. **Layer three: give the child its own cleanup deliberately.** If the child genuinely has something to clean up, drop the inherited entries with `atexit.unregister` first and register the child's own afterwards. Then the normal exit path in the child would be safe -- though the hard exit is still simpler and does not depend on anyone remembering. ### What not to reach for Do not "fix" it by removing the cleanup callback from the parent. The parent's release is real work; the bug is that a second process performed it. Do not fix it by widening the acquire timeout either -- that hides a correctness bug behind a longer wait, and on a long nightly run it converts a fast failure into a slow one. And do not assume that because the job usually succeeds the race is rare enough to live with: a lease released by a process that does not own it can also mean two processes reconciling the same batch, which is a data problem rather than a scheduling one. ### Version note The mechanics here are stable across Python 3.10 through 3.14. What changed around them is the default that steers people away from raw forking: since 3.14 the `multiprocessing` default start method is `forkserver` on Unix other than macOS, with macOS and Windows on `spawn` and `fork` requested explicitly. A job that has been calling `os.fork()` directly since long before that default moved is exactly the kind of code where this bug is still living.