Why does Python's print output appear instantly in a terminal but stall when the process is piped?
answer
- Depends on what stdout is attached to
- Terminal versus pipe behave differently
- One flush per newline, or per block
- A whole block before anything moves
- isatty decides it at interpreter startup
basics
~10 sCPython picks buffering per stream at startup: a terminal gets line buffering, so every newline flushes; a pipe or file gets a block buffer, 128 KiB on Python 3.14, that waits until it fills.
solid answer
~40 sAt interpreter startup CPython builds `sys.stdout` as a text wrapper over a buffered binary writer, and it asks the underlying descriptor whether it is a terminal. If `sys.stdout.isatty()` is true the wrapper is created with `line_buffering=True`, so each newline triggers a flush and you see output as it is produced. If stdout is a pipe or a file it is block buffered instead: writes accumulate in a buffer of `io.DEFAULT_BUFFER_SIZE`, which is 128 KiB on Python 3.14 and was 8 KiB through 3.13, and only reach the descriptor when that fills, when something calls `flush()`, or when the interpreter flushes the standard streams during a normal shutdown. Nothing is lost on a clean exit, it just arrives late and in one lump. `sys.stderr` is the exception: it is line buffered even when redirected.
code
python · 6 linesimport io
import sys
print("isatty:", sys.stdout.isatty())
print("line_buffering:", sys.stdout.line_buffering)
print("block size:", io.DEFAULT_BUFFER_SIZE)go deeper
Be ready to say the two modes out loud: line buffered on a terminal, block buffered on a pipe or file, chosen at startup. Knowing that print(..., flush=True) exists is the expected first fix.
Explain the mechanics: a text wrapper over a buffered writer, line_buffering set from isatty(), a block of io.DEFAULT_BUFFER_SIZE bytes, and the four things that drain it, including the flush during normal interpreter shutdown.
Show the diagnosis. Distinguish late output from lost output, connect loss to shutdown paths that skip finalisation, and pick the fix that belongs in the producer rather than the log collector.
Own the policy: whether services set unbuffered output everywhere and accept the syscall cost, or standardise on line buffering, and how that interacts with how logs are captured across a fleet.
## Two layers, one decision made at startup `sys.stdout` is not a raw file descriptor. It is a three-layer stack: a raw unbuffered file object on descriptor 1, an `io.BufferedWriter` on top of it, and an `io.TextIOWrapper` on top of that which does the `str`-to-`bytes` encoding. Buffering lives in the middle layer, and how the top layer behaves is fixed when the interpreter creates the stream, before your first line of code runs. The decision is made per stream, by asking the descriptor whether it is a terminal: - **Interactive (a tty).** The text wrapper is built with `line_buffering=True`. Every write that contains a newline flushes the buffer straight down to the descriptor. A human is watching, so latency matters more than syscall count. - **Not a tty (a pipe, a file, a socket, a container's captured stream).** The wrapper is built with `line_buffering=False`, and the buffered writer holds bytes until it has accumulated a block, `io.DEFAULT_BUFFER_SIZE`, which is 131072 bytes on Python 3.14 and was 8192 through 3.13. Nobody is watching interactively, so CPython optimises for throughput: one `write(2)` per block instead of one per line. That single branch explains the whole phenomenon. Run a script in a terminal and progress lines appear one at a time. Run the same script through a pipe, into a file, or under a process manager that captures output, and you see nothing for a long time, then a burst of hundreds of lines at once. ## What actually flushes a block-buffered stream Four things: 1. **The buffer fills.** Once a whole block of encoded bytes has accumulated, the writer drains itself. 2. **An explicit flush.** `sys.stdout.flush()`, or `print("x", flush=True)`, which calls it for you. 3. **Normal interpreter shutdown.** CPython flushes the standard streams as it finalises, so a script that runs to completion, or exits via `sys.exit()`, or dies of an uncaught exception, still delivers everything it wrote. This is why the output is late, not missing. 4. **Closing the stream.** The corollary is the dangerous case: shutdown paths that skip finalisation lose the buffer. `os._exit()` bypasses it by design, and so does being killed by a signal whose default disposition is to terminate the process, such as `signal.SIGKILL`, or an unhandled `signal.SIGTERM`. Whatever was still sitting in the block buffer is gone, and the last thing you see in the log is not the last thing the program did. ## stderr is deliberately different `sys.stderr` is line buffered **even when it is not a terminal**, and has been since Python 3.9. Diagnostics are supposed to escape the process promptly, so the stream that carries them does not sit on a block buffer. That asymmetry is useful to remember: if a program's tracebacks show up in a redirected log but its `print()` progress lines do not, you are looking at the difference between a line-buffered stream and a block-buffered one, not at a program that stopped early. ## Checking rather than guessing Both facts are introspectable at runtime. `sys.stdout.isatty()` tells you which branch was taken, and `sys.stdout.line_buffering` reports the mode the wrapper was actually built with. Running the same one-liner with and without a trailing `| cat` is the fastest way to see the switch happen. ```python import sys print(sys.stdout.isatty(), sys.stdout.line_buffering) ``` ## Why the design is right, and where it bites A syscall per line would be a real cost for a program that writes a million lines into a file, and the buffering is invisible when the consumer is a file you read after the fact. It bites in exactly one situation: when a human or a collector is reading the pipe *while the program runs*. Then the default optimises for the wrong thing, and you have to say so explicitly, either from the outside with the `-u` flag or the `PYTHONUNBUFFERED` environment variable, or from the inside with `flush=True` on the writes that matter or `sys.stdout.reconfigure(line_buffering=True)` early in `main()`. The common junior mistake is to conclude the program hung. It did not; the program is ahead of its output. The second mistake is to assume the shell or the pipe is buffering. The pipe has a kernel buffer of its own, but it is the Python process's userspace buffer that is holding your lines, which is why the fix belongs in the producer and not in the consumer.
- If the output is only delayed, when is it actually lost?When the process never reaches normal interpreter shutdown. `os._exit()` skips the flush by design, and so does termination by a signal with no handler, such as `signal.SIGKILL` or an unhandled `signal.SIGTERM`, or a hard crash of the process. Anything still in the block buffer dies with the process, so the tail of the log is missing even though the program got further than the log shows.
- Does the same rule apply to sys.stdin?`sys.stdin` is always buffered, regardless of whether it is a terminal, because the text layer needs a buffered binary stream underneath it to read ahead. That is also why `-u` and `PYTHONUNBUFFERED` are documented as affecting only stdout and stderr; input buffering is not something you turn off through those switches.
- Why does a child process launched from Python show the same symptom even harder?Because the child's stdout is a pipe you created, so the child takes the block-buffered branch too. If the child is itself a Python program you fix it at the child's end, by launching it with the `-u` flag or with `PYTHONUNBUFFERED=1` in the environment you pass it; reading faster in the parent cannot pull out bytes the child has not written yet.
A terminal-attached stream is a waiter bringing each dish as it is plated; a piped stream is one who waits until the whole tray is full, then carries eight dishes out at once.
saying these in an interview costs you the question
- Says the program hung when it is only buffering
- Claims the shell or the pipe is holding the lines
- Thinks buffered output is lost on a clean exit
- Believes print always writes straight to the descriptor
- Assumes stderr behaves the same as stdout when redirected
- Tries to fix it in the reading process, not the writer