External Commands with subprocess

`subprocess.run()` is the high-level entry point added in Python 3.5 and is the call you should reach for by default. `subprocess.call`, `check_call` and `check_output` are older thin wrappers over the same machinery, and `os.system` hands a string to a shell and gives you back only a status code with no access to output. `run()` constructs a `subprocess.Popen`, waits for the child process to exit, and returns a `subprocess.CompletedProcess`. ## What CompletedProcess carries Exactly four attributes: `args` (whatever you passed in), `returncode`, `stdout` and `stderr`. The last two are `None` unless you asked for redirection, and that is the single most common surprise for people new to the module. `capture_output=True` is shorthand for `stdout=subprocess.PIPE, stderr=subprocess.PIPE`. You can also redirect a single stream: `stderr=subprocess.STDOUT` merges the child's error stream into its output stream so you get one interleaved text; `stdout=subprocess.DEVNULL` throws output away. Passing `capture_output=True` together with an explicit `stdout=` or `stderr=` raises `ValueError` - pick one style. ## bytes or str By default the pipes are binary and both attributes come back as `bytes`. `text=True` (added in 3.7; the older spelling `universal_newlines=True` still works) decodes with the locale encoding and turns on universal newlines, so `\r\n` from a Windows-flavoured child arrives as `\n`. When the child's encoding is known, say so explicitly with `encoding="utf-8"`, and consider `errors="replace"` for output you only intend to log. Depending on the ambient locale is how a job that works on a developer laptop dies with `UnicodeDecodeError` inside a minimal container image. The `input=` argument sends data to the child's stdin and then closes it; it must be `bytes` unless text mode is on. ## Exit status is not an exception This is the part interviewers actually probe. A child that exits 1 produces a perfectly ordinary `CompletedProcess`. Code that writes `subprocess.run([...])` and never looks at the result has accepted the failure. `check=True` converts a non-zero exit into `subprocess.CalledProcessError`, and that exception carries `returncode`, `cmd`, `stdout` and `stderr` - which is why `check=True` and `capture_output=True` belong together: the traceback then contains the child's own diagnostic instead of a bare exit code. If you would rather inspect the result first, call `check_returncode()` on the `CompletedProcess` to raise the same exception on demand. On POSIX, a negative `returncode` of `-N` means the child was terminated by signal number N rather than exiting on its own. Starting the program is a separate failure mode from running it. With the default `shell=False`, a program that is not on the path raises `FileNotFoundError` (a subclass of `OSError`) - there is no exit code at all, because nothing ever ran. A hardened call site therefore expects three shapes of failure: `OSError` for launch problems, `CalledProcessError` for a bad exit code, and `subprocess.TimeoutExpired` when you passed `timeout=`. ## A concrete miss A nightly search-index rebuilder invokes a helper script that dies at startup because two of its own modules import each other. Python prints an `ImportError` traceback to stderr and exits with status 1. Without `check=True`, the rebuilder receives a `CompletedProcess`, never reads `returncode`, treats the step as done and publishes an empty index; the failure was loud in the child and completely silent in the parent. With `check=True, capture_output=True, text=True`, a `CalledProcessError` propagates and its `stderr` holds the traceback that explains it. ## Everyday neighbours of these arguments `timeout=` bounds the wall-clock wait. `cwd=` runs the child in another directory without mutating the parent's process-wide working directory, which `os.chdir` would do for every thread at once. `env=` replaces the child's environment wholesale rather than adding to it, so build it from `os.environ.copy()` and mutate that copy unless you genuinely want a bare environment. Finally, remember that `capture_output` buffers the entire output in memory: for a child that emits hundreds of megabytes, redirect `stdout=` to an open file object, or drop down to `Popen` and stream it. Everything described here is current behaviour on CPython 3.14.