What does os.set_inheritable control, and which descriptors does a child process actually inherit?

seniornice to knowfreq 20%

answer

Two ways a child is born
The flag is about the new program image
Copying a process copies the table
Python opted out of inheriting by default
Dirty buffers cross the fork too

basics

~10 s

It sets whether a descriptor survives exec into a new program image. It does not affect os.fork, which copies the entire descriptor table. Since Python 3.4 every descriptor Python creates is non-inheritable by default.

solid answer

~40 s

os.set_inheritable(fd, flag) and os.get_inheritable(fd) control the kernel's close-on-exec behaviour: a non-inheritable descriptor is closed when the process execs a new program. It says nothing about os.fork(), which copies the whole descriptor table, so a forked child that has not exec'd holds copies of everything the parent had. Since Python 3.4 (PEP 446) descriptors created by Python — open(), socket.socket(), os.pipe() — are non-inheritable by default, which stopped children silently holding listening sockets and log files open. subprocess adds its own policy: descriptors above the three standard streams are closed in the child unless listed in pass_fds. multiprocessing depends on the start method — with fork the child inherits everything, and in 3.14 the default on Unix other than macOS became forkserver (macOS and Windows use spawn), so that inheritance is no longer the default.

`os.set_inheritable(fd, flag)` and `os.get_inheritable(fd)` control the kernel's close-on-exec behaviour: a non-inheritable descriptor is closed when the process **execs** a new program. It says nothing about `os.fork()`, which copies the whole descriptor table, so a forked child that has not exec'd holds copies of everything the parent had. Since Python 3.4 (PEP 446) descriptors created by Python — `open()`, `socket.socket()`, `os.pipe()` — are non-inheritable by default, which stopped children silently holding listening sockets and log files open. `subprocess` adds its own policy: descriptors above the three standard streams are closed in the child unless listed in `pass_fds`. `multiprocessing` depends on the start method — with `fork` the child inherits everything, and in 3.14 the default on Unix other than macOS became `forkserver` (macOS and Windows use `spawn`), so that inheritance is no longer the default.

code

python · 8 lines

python

import os
import socket

s = socket.socket()
print(os.get_inheritable(s.fileno()))  # False by default since Python 3.4
os.set_inheritable(s.fileno(), True)
print(os.get_inheritable(s.fileno()))  # now survives an exec in a child
s.close()

go deeper

for a junior

Know that a child process can end up holding the parent's open files and sockets, and that Python has explicit functions to control it. Depth is not expected at this level.

for a middle

Be able to state the distinction that matters: the inheritable flag applies at exec, while fork copies the whole descriptor table regardless. Know that Python creates descriptors non-inheritable by default.

for a senior

Show the production consequences: a child holding a listening socket that cannot be released, and duplicated output when unflushed buffers cross a fork. Know how subprocess and each multiprocessing start method differ in what they pass through.

for a principal

Own the choice of start method as a platform decision - what a worker is allowed to inherit, why exec-based starts are the safer default for pools, and what that costs in startup time and in how state must be passed explicitly.

Two different mechanisms, routinely confused

"Does the child get my descriptors?" has two answers in Python, because there are two ways a child comes into existence.

os.fork() copies the whole process, and that includes the entire file-descriptor table. The child gets a copy of every open descriptor the parent had — regular files, sockets, pipes, everything — regardless of any flag. Nothing is filtered.

Executing a new program (what subprocess does, and what a fork is usually followed by) is where the inheritable flag applies. A descriptor marked non-inheritable carries the kernel's close-on-exec flag, so the kernel closes it as the new program image is loaded. The Python-level controls are os.get_inheritable(fd) and os.set_inheritable(fd, True|False); sockets additionally expose the same pair via their own methods.

So os.set_inheritable governs exec, not fork. A descriptor you mark non-inheritable is still present in a forked child that has not exec'd anything.

The Python default, and why it changed

Since Python 3.4 (PEP 446), every descriptor created by Python is non-inheritable by default — open(), socket.socket(), os.pipe(), tempfile and the rest. Before that, descriptors inherited the C default of being passed on, which caused two chronic bugs: a child process holding a copy of a listening socket or a log file it had no business owning (so the file could not be released even after the parent closed it), and a security problem where a descriptor to something sensitive was handed to a program that was never meant to see it. The current default fails safe; you opt in per descriptor when a child genuinely needs one.

subprocess layers its own policy on top: by default it closes descriptors above the three standard streams in the child, and pass_fds is the explicit way to hand specific ones through (which also forces them inheritable). The stdin/stdout/stderr arguments are handled separately, because those descriptors are placed onto 0, 1 and 2 in the child and must survive the exec by design.

multiprocessing depends entirely on the start method. With fork, the child begins as a copy of the parent and holds copies of all its descriptors. With spawn and forkserver, a fresh interpreter is started and only what the implementation explicitly passes gets through. In 3.14 the default start method on Unix other than macOS became forkserver; macOS and Windows already defaulted to spawn, and fork must now be requested explicitly. The practical effect is that the "my worker mysteriously holds the parent's socket open" class of bug is no longer the default experience on Linux.

The duplicated-side-effect trap

The nastiest consequence of fork inheritance is not the descriptor itself but the userspace buffer attached to it. Consider a thumbnail worker that appends a line to an output log and then forks a child to do the resize:

The line the parent wrote is still sitting in the file object's buffer when fork() runs. The child receives a copy of that buffer and a copy of the descriptor pointing at the same open file description. When each process later flushes — at close, or at exit — the same bytes are written twice. The result is a log or an output file with duplicated records that no single code path explains, and it scales with fork rate: the more work, the more duplication.

The remedies are all about not carrying dirty buffers across the fork: flush (or close) every buffered writer before forking, do the work in the parent and let children write to their own files, or use a start method that execs a fresh interpreter, which is precisely what spawn and forkserver give you. The same reasoning applies to anything else copied wholesale — a database connection, an SSL session, a random-number generator state — which is the deeper reason fork without exec is a poor default for a worker pool.

What to check when it matters

os.get_inheritable(fd) on a descriptor tells you exactly what will happen at exec; a census of /proc/self/fd inside a child tells you what actually came across. And when the goal is simply "the child should not see any of this", the modern answer is not to hand-manage flags at all, but to launch children in a way that starts a clean interpreter.

### Two different mechanisms, routinely confused "Does the child get my descriptors?" has two answers in Python, because there are two ways a child comes into existence. **`os.fork()` copies the whole process**, and that includes the entire file-descriptor table. The child gets a copy of every open descriptor the parent had — regular files, sockets, pipes, everything — regardless of any flag. Nothing is filtered. **Executing a new program** (what `subprocess` does, and what a fork is usually followed by) is where the *inheritable* flag applies. A descriptor marked non-inheritable carries the kernel's close-on-exec flag, so the kernel closes it as the new program image is loaded. The Python-level controls are `os.get_inheritable(fd)` and `os.set_inheritable(fd, True|False)`; sockets additionally expose the same pair via their own methods. So `os.set_inheritable` governs *exec*, not *fork*. A descriptor you mark non-inheritable is still present in a forked child that has not exec'd anything. ### The Python default, and why it changed Since Python 3.4 (PEP 446), **every descriptor created by Python is non-inheritable by default** — `open()`, `socket.socket()`, `os.pipe()`, `tempfile` and the rest. Before that, descriptors inherited the C default of being passed on, which caused two chronic bugs: a child process holding a copy of a listening socket or a log file it had no business owning (so the file could not be released even after the parent closed it), and a security problem where a descriptor to something sensitive was handed to a program that was never meant to see it. The current default fails safe; you opt in per descriptor when a child genuinely needs one. `subprocess` layers its own policy on top: by default it closes descriptors above the three standard streams in the child, and `pass_fds` is the explicit way to hand specific ones through (which also forces them inheritable). The `stdin`/`stdout`/`stderr` arguments are handled separately, because those descriptors are placed onto 0, 1 and 2 in the child and must survive the exec by design. `multiprocessing` depends entirely on the start method. With `fork`, the child begins as a copy of the parent and holds copies of all its descriptors. With `spawn` and `forkserver`, a fresh interpreter is started and only what the implementation explicitly passes gets through. **In 3.14 the default start method on Unix other than macOS became `forkserver`**; macOS and Windows already defaulted to `spawn`, and `fork` must now be requested explicitly. The practical effect is that the "my worker mysteriously holds the parent's socket open" class of bug is no longer the default experience on Linux. ### The duplicated-side-effect trap The nastiest consequence of fork inheritance is not the descriptor itself but the *userspace buffer attached to it*. Consider a thumbnail worker that appends a line to an output log and then forks a child to do the resize: The line the parent wrote is still sitting in the file object's buffer when `fork()` runs. The child receives a copy of that buffer and a copy of the descriptor pointing at the same open file description. When each process later flushes — at close, or at exit — the same bytes are written twice. The result is a log or an output file with duplicated records that no single code path explains, and it scales with fork rate: the more work, the more duplication. The remedies are all about not carrying dirty buffers across the fork: flush (or close) every buffered writer before forking, do the work in the parent and let children write to their own files, or use a start method that execs a fresh interpreter, which is precisely what `spawn` and `forkserver` give you. The same reasoning applies to anything else copied wholesale — a database connection, an SSL session, a random-number generator state — which is the deeper reason `fork` without `exec` is a poor default for a worker pool. ### What to check when it matters `os.get_inheritable(fd)` on a descriptor tells you exactly what will happen at exec; a census of `/proc/self/fd` inside a child tells you what actually came across. And when the goal is simply "the child should not see any of this", the modern answer is not to hand-manage flags at all, but to launch children in a way that starts a clean interpreter.

Why did Python change the default to non-inheritable in 3.4?
Because inheriting by default caused two chronic bugs. A child that exec'd an unrelated program kept copies of the parent's listening sockets and log files, so those resources could not be released even after the parent closed them; and a descriptor to something sensitive could reach a program that had no business seeing it. Making the default fail-safe and requiring an explicit opt-in per descriptor removed both.
The same log line appears twice in a file after a worker forks. What happened?
The parent had written the line into the file object's userspace buffer but not flushed it. `os.fork()` copied that buffer along with the descriptor, so both processes later flushed the same bytes to the same open file. The fix is to flush or close buffered writers before forking, or to use a start method that execs a fresh interpreter, such as spawn or forkserver.
How does a subprocess get a descriptor you deliberately want it to have?
List it in `pass_fds` on the `subprocess` call, which keeps it open across the exec and marks it inheritable for you. The `stdin`, `stdout` and `stderr` arguments are handled separately, since those are placed onto descriptors 0, 1 and 2 in the child by design. Everything else above the standard streams is closed in the child by default.

saying these in an interview costs you the question

Says os.set_inheritable controls what a fork child receives
Assumes Python descriptors are inheritable by default
Thinks fork filters descriptors by any flag
Believes multiprocessing children never inherit descriptors
Forgets that buffered data is duplicated across a fork

What does os.set_inheritable control, and which descriptors does a child process actually inherit?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Two different mechanisms, routinely confused

The Python default, and why it changed

The duplicated-side-effect trap

What to check when it matters

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions8 questions

▾related topics3 topics

Two different mechanisms, routinely confused

The Python default, and why it changed

The duplicated-side-effect trap

What to check when it matters

answer

go deeper

context