Inherited Environment and cwd · Python — interview questions

Does subprocess.run's env argument merge with the parent environment or replace it?

middlemust knowfreq 55%

answer

Not a patch
The child gets exactly what you pass
Missing PATH, HOME and locale
Merge with {**os.environ, ...}

basics

~20 s

It replaces it. The mapping you pass becomes the child's entire environment, so anything you leave out - PATH, HOME, LANG - is simply not there. Merge on purpose with a dict such as {**os.environ, 'KEY': 'value'}.

solid answer

~50 s

The env argument is not a patch; it is the whole environment block handed to the child. Leave it as None and the child inherits a copy of the parent's environment as it stands at spawn time. Pass a mapping and that mapping is all the child gets, which is why a command that works interactively fails inside a job runner as soon as somebody writes env={'API_TOKEN': tok}: no PATH, no HOME, no locale. On POSIX the executable search itself uses the PATH from the mapping you passed - CPython builds the candidate list with os.get_exec_path(env), which falls back to os.defpath when there is no PATH key - so a bare command name outside /bin and /usr/bin raises FileNotFoundError before the program ever starts. Merge deliberately, or build a minimal environment on purpose and put PATH in it yourself. Keys and values must be strings.

The `env` argument is not a patch; it is the whole environment block handed to the child. Leave it as `None` and the child inherits a copy of the parent's environment as it stands at spawn time. Pass a mapping and that mapping is all the child gets, which is why a command that works interactively fails inside a job runner as soon as somebody writes `env={'API_TOKEN': tok}`: no `PATH`, no `HOME`, no locale. On POSIX the executable search itself uses the `PATH` from the mapping you passed - CPython builds the candidate list with `os.get_exec_path(env)`, which falls back to `os.defpath` when there is no `PATH` key - so a bare command name outside `/bin` and `/usr/bin` raises `FileNotFoundError` before the program ever starts. Merge deliberately, or build a minimal environment on purpose and put `PATH` in it yourself. Keys and values must be strings.

code

python · 11 lines

python

import os
import subprocess
import sys

os.environ["PICK_WAVE"] = "17"
show = [sys.executable, "-c",
        "import os; print(os.environ.get('PICK_WAVE'), len(os.environ))"]

subprocess.run(show, check=True)                                    # inherited copy
subprocess.run(show, env={"PICK_WAVE": "18"}, check=True)           # replaced
subprocess.run(show, env={**os.environ, "PICK_WAVE": "18"}, check=True)  # merged

go deeper

for a junior

Recall the one-line rule and be able to write the fix: no env argument means the child inherits a copy of your environment; an env argument replaces it completely, so merge with {**os.environ, 'KEY': 'value'} when you only meant to add something.

for a middle

Explain the mechanics: the mapping is flattened into NAME=VALUE strings for the exec call, values must be strings, and on POSIX the program search itself uses the PATH from that mapping, falling back to os.defpath. That is why the symptom is FileNotFoundError.

for a senior

Show the production instinct - build the child's environment explicitly instead of inheriting whatever a scheduler happened to export, resolve the executable to an absolute path first, and keep the required variable set documented so a missing HOME or LANG is a known list rather than a two-hour outage.

for a principal

Own the policy: which variables may cross a process boundary at all, whether jobs inherit ambient credentials or receive a curated block, and how that rule is enforced in a shared runner so no team ever again debugs a difference between one machine's PATH and another's.

What an environment actually is. A process carries an environment block: a flat set of NAME=VALUE strings handed to a program at the moment it is executed. There is no live link afterwards. The child gets a copy at exec time, and from that instant the two processes drift apart - nothing the parent does to its own environment later reaches a child that is already running, and nothing the child does comes back.

What env=None means. When you call subprocess.run(argv) with no env, CPython does not build an environment block at all. The child is launched in a way that lets it inherit the parent's current C-level environment wholesale. That is why a variable set with os.environ['PICK_WAVE'] = '17' - an assignment that updates the mapping and pushes the value down into the C environment for you - is visible to a child spawned on the next line.

What env=<mapping> means. The instant you pass a mapping, that mapping becomes the entire block. It is a replacement, not an update, and the failure it causes is famous because it does not look like an environment problem:

python

subprocess.run(['reportgen', 'today'], env={'API_TOKEN': tok})
# FileNotFoundError: [Errno 2] No such file or directory: 'reportgen'

The program is installed and on your PATH; the child just never got a PATH. On POSIX, subprocess resolves a bare command name by building a candidate list from os.get_exec_path(env), which reads PATH out of the mapping you supplied and, finding none, falls back to os.defpath - :/bin:/usr/bin on a typical Unix. Anything installed outside those two directories is invisible. Drop HOME and tools write their state somewhere unexpected or refuse to start; drop LANG and LC_ALL and the child runs in the C locale, which quietly changes its output encoding and its sort order; drop TMPDIR and temporary files land on a different filesystem. On Windows, a great many programs need SystemRoot present in the block, and the operating system performs the executable search with the parent's PATH, so a PATH you put in env does not redirect which file is found there.

The merge idiom. Build the child's environment from the parent's, explicitly:

python

env = {**os.environ, 'PICK_WAVE': '17'}
subprocess.run(argv, env=env, check=True)

Later keys win, so overriding is natural and removing a variable means omitting it from the merged dict - not setting it to None or to an empty string, which produces a variable that exists and is empty, a state many programs treat differently from absent.

Or replace on purpose. A curated block is a legitimate and often better choice for a job runner: decide which variables cross the boundary, put PATH, HOME, LANG and whatever the program documents into a dict, and hand that over. Combine it with resolving the program yourself - shutil.which returns an absolute path - so the spawn does not depend on the child's PATH at all. That turns a machine-specific mystery into a value you can log.

Typing rules. The mapping is flattened into NAME=VALUE strings for the exec call, so keys and values must be str (or bytes); an int or None raises TypeError, and an embedded NUL byte is rejected outright. Convert numbers with str() at the call site rather than relying on any coercion, because there is none.

What env is not. It is not a channel. A running child never sees a later change, so environment variables are startup configuration, not signalling. And in a parent that spawns from several threads, mutating os.environ per job is the wrong tool for exactly the same reason os.chdir is: it is process-global state, and whichever value happens to be set when a thread reaches its exec is the value that child receives. Per-job variation belongs in the env argument, one dict per spawn.

How to debug it in ten seconds. Spawn the interpreter itself with the same env and cwd you would give the real helper and print what arrives:

python

subprocess.run([sys.executable, '-c', 'import os; print(sorted(os.environ))'], env=env)

If PATH is missing from that list, you have found the bug without touching the helper at all.

**What an environment actually is.** A process carries an environment block: a flat set of `NAME=VALUE` strings handed to a program at the moment it is executed. There is no live link afterwards. The child gets a *copy* at exec time, and from that instant the two processes drift apart - nothing the parent does to its own environment later reaches a child that is already running, and nothing the child does comes back. **What `env=None` means.** When you call `subprocess.run(argv)` with no `env`, CPython does not build an environment block at all. The child is launched in a way that lets it inherit the parent's current C-level environment wholesale. That is why a variable set with `os.environ['PICK_WAVE'] = '17'` - an assignment that updates the mapping *and* pushes the value down into the C environment for you - is visible to a child spawned on the next line. **What `env=<mapping>` means.** The instant you pass a mapping, that mapping becomes the entire block. It is a replacement, not an update, and the failure it causes is famous because it does not look like an environment problem: ```python subprocess.run(['reportgen', 'today'], env={'API_TOKEN': tok}) # FileNotFoundError: [Errno 2] No such file or directory: 'reportgen' ``` The program is installed and on your `PATH`; the child just never got a `PATH`. On POSIX, `subprocess` resolves a bare command name by building a candidate list from `os.get_exec_path(env)`, which reads `PATH` out of the mapping you supplied and, finding none, falls back to `os.defpath` - `:/bin:/usr/bin` on a typical Unix. Anything installed outside those two directories is invisible. Drop `HOME` and tools write their state somewhere unexpected or refuse to start; drop `LANG` and `LC_ALL` and the child runs in the C locale, which quietly changes its output encoding and its sort order; drop `TMPDIR` and temporary files land on a different filesystem. On Windows, a great many programs need `SystemRoot` present in the block, and the operating system performs the executable search with the *parent's* `PATH`, so a `PATH` you put in `env` does not redirect which file is found there. **The merge idiom.** Build the child's environment from the parent's, explicitly: ```python env = {**os.environ, 'PICK_WAVE': '17'} subprocess.run(argv, env=env, check=True) ``` Later keys win, so overriding is natural and removing a variable means omitting it from the merged dict - not setting it to `None` or to an empty string, which produces a variable that exists and is empty, a state many programs treat differently from absent. **Or replace on purpose.** A curated block is a legitimate and often better choice for a job runner: decide which variables cross the boundary, put `PATH`, `HOME`, `LANG` and whatever the program documents into a dict, and hand that over. Combine it with resolving the program yourself - `shutil.which` returns an absolute path - so the spawn does not depend on the child's `PATH` at all. That turns a machine-specific mystery into a value you can log. **Typing rules.** The mapping is flattened into `NAME=VALUE` strings for the exec call, so keys and values must be `str` (or `bytes`); an `int` or `None` raises `TypeError`, and an embedded NUL byte is rejected outright. Convert numbers with `str()` at the call site rather than relying on any coercion, because there is none. **What `env` is not.** It is not a channel. A running child never sees a later change, so environment variables are startup configuration, not signalling. And in a parent that spawns from several threads, mutating `os.environ` per job is the wrong tool for exactly the same reason `os.chdir` is: it is process-global state, and whichever value happens to be set when a thread reaches its exec is the value that child receives. Per-job variation belongs in the `env` argument, one dict per spawn. **How to debug it in ten seconds.** Spawn the interpreter itself with the same `env` and `cwd` you would give the real helper and print what arrives: ```python subprocess.run([sys.executable, '-c', 'import os; print(sorted(os.environ))'], env=env) ``` If `PATH` is missing from that list, you have found the bug without touching the helper at all.

How would you give a child a deliberately minimal environment without breaking it?
List the variables the program actually needs rather than guessing: `PATH`, `HOME`, `LANG` or `LC_ALL`, `TZ`, plus anything the program documents (`SystemRoot` on Windows). Build that dict explicitly and resolve the executable with `shutil.which` up front so you can pass an absolute path, which takes the child's `PATH` out of the equation entirely. Log the keys you pass, so a future outage is one grep away instead of a bisect.
What happens if a value in the mapping is not a string?
It raises `TypeError` at spawn time. The mapping is flattened into `NAME=VALUE` strings for the exec call, so an `int`, a `None` or a path object is not accepted, and an embedded NUL byte is rejected as well. Convert at the call site with `str()`. The error surfaces from the spawn, not from the program, which is a useful clue: the child never ran.
Does a running child see a variable the parent sets after it started?
No. The environment is copied once, at exec. Changing `os.environ` in the parent afterwards affects only the parent and any child spawned from that point on. If a long-running child needs to be told something new, use a real channel - a pipe, a file it watches, a signal - and treat environment variables as startup configuration only.

Passing env is handing someone a fresh wallet rather than slipping one card into theirs: whatever you did not put in it is not in it.

saying these in an interview costs you the question

Thinks env updates the parent environment rather than replacing it
Assumes PATH is inherited even when a custom env omits it
Blames the program for FileNotFoundError after passing a one-key env
Sets a variable to an empty string to remove it
Passes ints or None as environment values
Believes a running child sees later changes to os.environ

context

open as a page

How does subprocess.run find the program when args[0] is a bare command name?

juniorshould knowfreq 42%

basics

~20 s

With shell=False, a name containing no directory separator is looked up in the PATH directories in order and the first match runs. A name with a separator is a path, resolved against the child's working directory. shutil.which runs the same search.

open as a page

How does os.environ relate to the real process environment a child inherits?

middleshould knowfreq 38%

basics

~20 s

os.environ is a dict-like snapshot taken when the os module is first imported. Writing to it also pushes the value into the C environment, so children spawned afterwards see it; os.putenv changes only the C side and leaves the mapping stale.

open as a page

Why is os.chdir the wrong way to point each subprocess.run child at its own directory in a threaded worker?

seniorshould knowfreq 35%

basics

~20 s

The working directory is process-global, not per thread, so concurrent workers overwrite each other's value and a child can start in another job's directory. Pass cwd= to subprocess.run instead: that change of directory happens inside the child, once per spawn.

open as a page

§explore

▾roadmaps that cover this13

§questions

Does subprocess.run's env argument merge with the parent environment or replace it?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

▾common follow-ups3 questions

▾analogy~10 s

▾red flags6 flags

§ context

▾related questions5 questions

▾related topics4 topics

How does subprocess.run find the program when args[0] is a bare command name?

basics

How does os.environ relate to the real process environment a child inherits?

basics

Why is os.chdir the wrong way to point each subprocess.run child at its own directory in a threaded worker?

basics

explore

questions

answer

go deeper

context