skip to content

Unix Runtime Behavior

Signals delivered between bytecodes, children that hang or go zombie, fork hazards, resource ceilings and output stuck in a buffer. It separates someone who scripts Python from someone who runs it.

part ofPythonoverview, primer and where to startread it →
on this pageshow

explore

questions

95 · 6 sections

How do you bound a blocking call with signal.alarm when it takes no timeout?

level: juniorimportance: must knowfreq 42%
basics
~10 s

Install a SIGALRM handler with signal.signal that raises an exception, call signal.alarm(seconds) just before the blocking call, and call signal.alarm(0) in a finally block. The handler's exception unwinds the blocked call.

open as a page

How do signal.SIGINT and signal.SIGTERM differ in a Python process by default?

level: juniorimportance: must knowfreq 60%
basics
~20 s

CPython installs its own default handler for signal.SIGINT that raises KeyboardInterrupt in the main thread, so finally blocks and atexit hooks still run. signal.SIGTERM keeps the operating system default, which terminates the process immediately with no Python cleanup.

open as a page

What did PEP 475 change in Python 3.5 about system calls interrupted by a signal?

level: middleimportance: must knowfreq 30%
basics
~20 s

Since Python 3.5, the interpreter reissues a system call that a signal interrupted instead of raising InterruptedError, after running the Python-level handler and recomputing any remaining timeout. Hand-written EINTR retry loops around stdlib calls became dead code.

open as a page

Why can a handler installed with signal.signal() run well after the signal actually arrives?

level: middleimportance: must knowfreq 58%
basics
~10 s

CPython installs its own C handler, which only records that the signal is pending. Your Python function runs later, when the main thread of the main interpreter reaches its next bytecode boundary check.

open as a page

Why does a Python process running as PID 1 ignore signal.SIGTERM?

level: middleimportance: must knowfreq 45%
basics
~10 s

The kernel gives PID 1 no default action: a signal left at its default disposition is discarded, not acted on. CPython installs a handler for SIGINT but not SIGTERM, so call signal.signal(signal.SIGTERM, handler) yourself.

open as a page

How do you make subprocess.run raise on a failing command, and which exception carries the exit code?

level: juniorimportance: must knowfreq 60%
basics
~20 s

Pass check=True to subprocess.run, or call check_returncode() on the CompletedProcess it returns. Both raise subprocess.CalledProcessError, whose returncode attribute holds the child's exit status. Without check, a non-zero status comes back silently and is easy to ignore.

open as a page

How do subprocess.Popen.terminate() and Popen.kill() differ on Unix?

level: juniorimportance: must knowfreq 62%
basics
~20 s

Popen.terminate() sends SIGTERM, which the child may catch and use to shut down cleanly. Popen.kill() sends SIGKILL, which no process can catch, block or ignore, so the child dies immediately with no cleanup. Both signal only the direct child.

open as a page

Why can Popen.wait() deadlock when the child's stdout is subprocess.PIPE?

level: middleimportance: must knowfreq 65%
basics
~20 s

A kernel pipe holds a fixed amount, roughly 64 KiB on Linux. Once the child fills it, the child blocks in write() while the parent blocks in Popen.wait, so neither moves. Popen.communicate drains both pipes and waits in one call.

open as a page

Does subprocess.run's env argument merge with the parent environment or replace it?

level: middleimportance: must knowfreq 55%
basics
~20 s

It replaces it. The mapping you pass becomes the child's entire environment, so anything you leave out - PATH, HOME, LANG - is simply not there. Merge on purpose with a dict such as {**os.environ, 'KEY': 'value'}.

open as a page

Why does Popen.terminate() leave a shell=True command's real program running?

level: middleimportance: must knowfreq 58%
basics
~20 s

With shell=True the direct child is /bin/sh, so Popen.pid is the shell's pid and terminate() signals the shell, not the program it started. Start the child with start_new_session=True and signal the whole group with os.killpg(os.getpgid(p.pid), signal.SIGTERM).

open as a page

Why do Python pre-fork workers lose the copy-on-write sharing they start with?

level: middleimportance: must knowfreq 45%
basics
~20 s

Because CPython writes an object's reference count whenever a reference to it is taken or dropped. Merely reading a shared object dirties the whole page holding it, so a worker's private memory grows until little is really shared.

open as a page

Why should a child created by os.fork() exit with os._exit()?

level: middleimportance: must knowfreq 55%
basics
~10 s

sys.exit() only raises SystemExit, so the child unwinds, runs the parent's inherited atexit callbacks and flushes inherited buffers -- cleanup that happens twice. os._exit() ends the process immediately, skipping all of it.

open as a page

Why does CPython raise a DeprecationWarning for os.fork() in a multi-threaded process?

level: middleimportance: must knowfreq 42%
basics
~20 s

Because the child gets the parent's whole memory but only the forking thread, so any lock the vanished threads held stays locked forever and the child deadlocks. Python 3.12 added the warning; Python 3.14 moved multiprocessing's Unix default to forkserver.

open as a page

After os.fork(), does a change the child makes to a Python list reach the parent?

level: juniorimportance: should knowfreq 40%
basics
~20 s

No. os.fork() gives the child a copy-on-write copy of the parent's address space, so the child sees the parent's objects exactly as they were at fork time, but every write it makes stays private to it.

open as a page

Why does output written before os.fork() get printed twice?

level: juniorimportance: should knowfreq 40%
basics
~10 s

os.fork() copies the whole process, including text still sitting in stdout's userspace buffer. Both copies flush that same pending text later, so it prints twice. Call sys.stdout.flush() immediately before forking.

open as a page

Why can open() without a with block leak a file descriptor in Python?

level: juniorimportance: must knowfreq 70%
basics
~20 s

A file object holds a kernel descriptor until something closes it. CPython usually closes it when the last reference disappears, but that is a refcounting implementation detail. A with block closes deterministically, even when the body raises.

open as a page

What does os.cpu_count() report, and why is it the wrong number inside a container?

level: middleimportance: must knowfreq 55%
basics
~10 s

os.cpu_count() reports the logical processors the host kernel exposes. A container's CPU limit is enforced as scheduling bandwidth rather than by hiding processors, so the count still says 64 on a two-CPU budget.

open as a page

When does CPython raise a catchable MemoryError instead of the container being OOM-killed?

level: middleimportance: must knowfreq 41%
basics
~20 s

MemoryError is raised only when an allocation request fails immediately, so the interpreter regains control. A cgroup limit is charged when pages are touched, not when they are requested, so the kernel kills the process instead -- there is nothing to catch.

open as a page

Using resource.setrlimit, which limit changes are reversible and which are not?

level: middleimportance: must knowfreq 46%
basics
~20 s

A process may move its soft limit freely between zero and its hard limit, so those changes are reversible. Lowering the hard limit is one-way: raising it again needs privilege an ordinary process lacks, and the drop is inherited by children.

open as a page

Why does a containerized Python process exit with status 137 and print no traceback?

level: juniorimportance: should knowfreq 48%
basics
~20 s

Status 137 is 128 + 9: the kernel delivered SIGKILL, most often when the process crossed its container memory limit. SIGKILL cannot be caught, so no except, finally or atexit code runs and nothing is printed.

open as a page

Why do pre-fork Python application servers run several worker processes instead of one?

level: juniorimportance: must knowfreq 70%
basics
~20 s

Because one CPython process executes Python bytecode on one core at a time under the global interpreter lock. Forking several workers, each with its own interpreter and its own lock, puts real work on every core and isolates a crash.

open as a page

Why does preloading a Python app before forking workers save less memory than copy-on-write suggests?

level: middleimportance: must knowfreq 55%
basics
~20 s

Copy-on-write shares pages only until something writes to one. CPython stores a reference count inside every object header, so merely touching a preloaded object dirties its page and the sharing quietly decays over the worker's lifetime.

open as a page

How do you use fcntl.flock to guarantee only one copy of a Python job runs?

level: middleimportance: must knowfreq 40%
basics
~20 s

Open a fixed lockfile with os.open, then call fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB). A BlockingIOError means another copy holds it, so exit. Keep the descriptor open for the whole run; the kernel releases the lock when the process dies.

open as a page

Why should a long-lived Python service run in the foreground instead of double-fork daemonising itself?

level: middleimportance: must knowfreq 52%
basics
~20 s

A supervisor manages a process by being its parent. Staying in the foreground keeps the supervisor as the parent, so it knows the pid, reads the exit status, and captures whatever the process writes to sys.stdout and sys.stderr.

open as a page

How do Python's pre-fork, threaded and async worker models differ in concurrency held and memory cost?

level: middleimportance: must knowfreq 65%
basics
~20 s

A pre-fork worker holds one request and costs a whole interpreter. A thread holds one request and costs a stack. An async worker holds thousands of waiting connections in one thread — but only while nothing blocks.

open as a page

Why does Python's print output appear instantly in a terminal but stall when the process is piped?

level: juniorimportance: must knowfreq 65%
basics
~10 s

CPython picks buffering per stream at startup: a terminal gets line buffering, so every newline flushes; a pipe or file gets a block buffer, 128 KiB on Python 3.14, that waits until it fills.

open as a page

Why does CPython raise BrokenPipeError instead of dying on SIGPIPE like a C filter?

level: middleimportance: must knowfreq 45%
basics
~20 s

CPython sets SIGPIPE to signal.SIG_IGN while the interpreter starts up. With the signal ignored, the failing write returns errno.EPIPE to the caller instead of terminating the process, and Python raises BrokenPipeError so the code can react.

open as a page

What do Python's -u flag and PYTHONUNBUFFERED change about stdout and stderr?

level: middleimportance: must knowfreq 55%
basics
~20 s

Both force stdout and stderr to be unbuffered: the binary layer under each stream is created without a buffer and the text layer is write-through, so every write reaches the descriptor at once. Neither affects stdin.

open as a page

Why does contextlib.redirect_stdout miss output from a compiled extension?

level: middleimportance: must knowfreq 45%
basics
~20 s

contextlib.redirect_stdout only rebinds the Python object sys.stdout. Compiled code writes straight to file descriptor 1 and never consults sys.stdout, so its output bypasses the capture; redirecting the descriptor itself with os.dup2 is what catches it.

open as a page

What does BrokenPipeError mean when a Python script writes to a pipe whose reader has exited?

level: juniorimportance: should knowfreq 40%
basics
~20 s

BrokenPipeError says a write failed because nobody is reading the other end any more: the downstream process exited or closed its read descriptor. The kernel fails the write with errno.EPIPE (32) and Python turns that into this OSError subclass.

open as a page