skip to content

What happens to subprocess.Popen children when the Python parent exits first?

level: seniorimportance: nice to knowfreq 24%

answer

  1. The kernel does not clean anything up
  2. Only one field about them changes
  3. Someone else adopts them
  4. PID 1 or the nearest subreaper
  5. Python emits at most a ResourceWarning

basics

~20 s

They keep running. A child whose parent dies is an orphan and is reparented by the kernel to PID 1, or to the nearest ancestor marked as a subreaper. Python never kills them for you, so cleanup has to be explicit.

solid answer

~50 s

Nothing in CPython terminates a `subprocess.Popen` child when the parent exits. On Unix the orphaned child is adopted by PID 1 — or by the nearest ancestor registered as a subreaper, such as a user service manager — which becomes responsible for reaping it. The child keeps everything it inherited: its listening socket, file locks, temp files and the inherited ends of any pipes, so the classic symptom is a port that will not rebind or an output file still being written by a process nobody is tracking. At interpreter exit CPython may only emit a `ResourceWarning` that a child is still running. Cleanup is therefore your job: start children with `start_new_session=True`, and kill the group from a `finally` block and from a parent SIGTERM handler. Note the contrast — a `multiprocessing.Process` marked `daemon=True` *is* terminated when its parent exits; subprocess children are not.

code

python · 12 lines
python
import subprocess, sys, time

MIDDLE = """
import os, subprocess, sys
subprocess.Popen([sys.executable, "-c",
                  "import os, time; time.sleep(0.5); print('grandchild ppid now:', os.getppid())"])
print("middle pid:", os.getpid())
"""

p = subprocess.Popen([sys.executable, "-c", MIDDLE])
p.wait()
time.sleep(1.5)

go deeper

for a junior

Remember the headline: a child you started keeps running after your script ends unless you stop it yourself. Python does not tidy processes up the way it tidies files.

for a middle

Explain the mechanics: the kernel reparents the orphan to PID 1 or the nearest subreaper, it retains every descriptor and lock it held, and Popen's context manager waits rather than kills.

for a senior

Connect it to symptoms you have debugged — a port that will not rebind, a file still growing after a job finished — and to cleanup that survives exceptions and an external SIGTERM.

for a principal

Decide where lifecycle ownership lives: how much cleanup application code should attempt versus what you delegate to a supervisor, unit or container runtime that still works when the parent is SIGKILLed.

An orphan is a process whose parent has exited while it is still running. It is not an error state and the kernel does not clean it up: the process keeps running exactly as before, with one field changed. ## What the kernel does On Unix, every process needs a parent to receive its exit status, so when a parent dies the kernel reassigns its live children. Historically they all went to PID 1, the init process. On modern Linux they go to the nearest *ancestor* that has registered itself as a child subreaper — a user-level service manager or a session leader typically does this — and only to PID 1 if there is none. You can watch the change directly: a grandchild that prints `os.getppid()` before and after its own parent exits reports two different numbers, the second usually 1. The practical meaning is that reparenting is a bookkeeping change, not a cleanup. Nothing signals the orphan. It keeps its memory, its open descriptors, its file locks, its listening socket, its position in a file it was writing, and the write end of any pipe it inherited. That last one is why a supposedly-dead pipeline can leave a reader blocked: the pipe does not report end-of-file while any process still holds a write end open, and an orphan you forgot about is exactly such a process. ## What CPython does — which is nothing There is no hidden cleanup in `subprocess`. When the interpreter exits with a live `Popen` child, Python does not signal it, does not wait for it, and does not kill its group. The most you get is a `ResourceWarning` noting that a child is still running, and warnings of that category are not displayed by default. That surprises people because two nearby mechanisms behave differently. A `multiprocessing.Process` created with `daemon=True` *is* terminated when its parent exits — that is what the daemon flag means there. And a daemon *thread* is simply abandoned when the interpreter shuts down. Neither rule extends to `subprocess`: a `Popen` child is an independent OS process and outlives you by default. The context manager does not close this gap either. `with subprocess.Popen(...) as p:` closes the pipes on exit and then calls `wait()`. It guarantees descriptors are released and the child is reaped, not that it stopped — a child that never exits turns the block into an indefinite hang instead of leaving an orphan. ## Why it matters in production The symptoms rarely name themselves. A service redeploys and fails to bind its port because a child from the previous generation still holds it. A scheduled job's output file keeps growing after the job "finished", because an orphaned exporter is still writing it. Two runs of the same job corrupt each other, because the first run's child never stopped and both are appending. Machine load stays high with no owning parent in sight. Each of these is one leaked child from a shutdown path that only signalled the direct child, or from a parent that crashed or was SIGKILLed before it could clean up. Containers add a twist. Inside a container your application is often PID 1 itself, and orphans reparent to *it*. PID 1 has a special obligation to reap adopted children, and an application process that never calls a wait function will accumulate them as zombies. It also has special signal semantics: default signal dispositions do not apply to PID 1, so a naive process can ignore the very SIGTERM the runtime sends to stop it. This is the whole reason minimal init processes are commonly used as a container's entrypoint. ## Preventing them Three layers, in order of how much they cover. First, make the tree reachable at spawn time: `start_new_session=True` so the child leads its own process group and every descendant it forks joins that group, and cache the group id straight away. Second, clean up on every exit path you control. Put the group kill in a `finally`, so exceptions and early returns are covered, and install a parent SIGTERM handler that runs the same routine, so an orderly shutdown from outside is covered too. `atexit` alone is not enough: it runs on a normal interpreter exit but not when your parent is SIGKILLed or dies from a fault. Third, accept that the parent can vanish without warning, and put something above you that can still clean up: a process supervisor, a systemd unit with a kill mode that covers the whole control group, or a container runtime that tears the container's processes down together. That is the only layer that survives your own SIGKILL, which is precisely the case where none of your code runs. When you are hunting an existing leak, the process group is also the diagnostic: an orphan spawned inside a session you created still carries that session and group id, so listing processes by group tells you which run leaked it.

  • Why can an orphaned child keep a reader in the parent blocked even after the process you started has died?
    Because end-of-file on a pipe is reported only when the last write end is closed. An orphan that inherited the write descriptor still holds it open, so the reader keeps waiting on a pipe nobody will write to. The fix is the same one that stops the orphan existing — signal the whole group rather than just the direct child, so no descendant survives holding descriptors.
  • How is an orphan different from a zombie?
    An orphan is alive and has lost its parent; a zombie has already exited and is waiting for some parent to collect its exit status. The two connect through reparenting: an orphan that later exits becomes a zombie under whichever process adopted it, and that adopter is expected to reap it. PID 1 does so routinely, which is why orphan leaks show up as running processes rather than as accumulating zombies.
  • Does running your application as PID 1 in a container change any of this?
    Yes, in two ways. Orphans reparent to your process rather than to a system init, so it inherits the obligation to reap them or they accumulate as zombies. And PID 1 has special signal semantics — default dispositions do not apply — so a process that has not installed a SIGTERM handler can ignore the runtime's stop signal entirely. That is why a minimal init process is commonly used as the entrypoint.

saying these in an interview costs you the question

  • Says the kernel kills a child when its parent dies
  • Thinks Python terminates Popen children at interpreter exit
  • Confuses an orphan with a zombie
  • Believes a with-block kills the child on exit
  • Assumes atexit covers a parent that was SIGKILLed
  • Expects daemon semantics to apply to subprocess children

context