skip to content

In os/exec, what goes wrong if you call cmd.Wait() before finishing reads from cmd.StdoutPipe()?

level: middleimportance: must knowfreq 52%

answer

  1. Wait does more than wait for the exit
  2. somebody closes the read end
  3. the symptom is truncation, not a panic
  4. it hides until the output grows
  5. Start, drain to end of file, then Wait

basics

~20 s

cmd.Wait closes the pipes it created once the child exits, so unfinished reads lose whatever was still in flight and then fail against a closed file. Read the pipe to end of file first, then call cmd.Wait.

solid answer

~50 s

The documented rule is that it is incorrect to call `cmd.Wait` before all reads from `cmd.StdoutPipe()` have completed. `Wait` waits for the process to exit and then closes the parent's ends of the pipes it created, so a read still in flight — or any read you issue afterwards — meets a closed `*os.File` and returns an error instead of the remaining bytes. Whatever was still sitting in the kernel's pipe buffer is gone, and the usual symptom is silently truncated output rather than a crash. The correct sequence is `cmd.Start`, read to end of file, then `cmd.Wait`, and the same holds for `cmd.StderrPipe()`. If you would rather not own that ordering, do not take the pipe: assign a writer to `cmd.Stdout` and let os/exec's own copying goroutine do the draining, since `Wait` joins that goroutine before it returns.

code

go · 15 lines
go
stdout, err := cmd.StdoutPipe()
if err != nil {
	return err
}
if err := cmd.Start(); err != nil {
	return err
}
data, err := io.ReadAll(stdout)
if err != nil {
	return err
}
if err := cmd.Wait(); err != nil {
	return err
}
return process(data)

go deeper

for a junior

Recall the order and keep it fixed: start the command, read the pipe until end of file, and only then call cmd.Wait. Never call cmd.Wait first and read afterwards.

for a middle

Explain that cmd.Wait waits for the process and then closes the pipes it created, so unfinished reads lose whatever remained in the kernel buffer and then fail against a closed file.

for a senior

Show that you recognise intermittently truncated child output as an ordering bug rather than a child bug, and that draining two pipes needs concurrency, plus what cmd.WaitDelay is for when Wait hangs after exit.

for a principal

Frame it as an API-shape decision: whether the team's subprocess wrapper exposes raw pipes at all, and what it guarantees about output completeness and bounded waits.

## What cmd.Wait actually does `cmd.Wait` is not only "block until the child exits". It does three things in order: it waits for the process to terminate, it waits for os/exec's internal copying goroutines (the ones created when you assigned a non-file writer to `cmd.Stdout` or `cmd.Stderr`) to finish, and it closes the parent's ends of any pipes it created — including the ones handed to you by `cmd.StdoutPipe`, `cmd.StderrPipe` and `cmd.StdinPipe`. That last step is a convenience: it means you normally never have to `Close` the reader you were given. It is also the trap, because the closing is unconditional. `Wait` has no idea whether you finished reading. ## The failure, precisely Suppose the child writes 300 KB and exits. Much of that is still sitting in the kernel's pipe buffer or has not been read into your program yet. If you call `cmd.Wait` at that moment: - the process has exited, so `Wait` proceeds; - it closes the parent's read end; - your subsequent `Read` returns an error indicating the file is already closed, matching `os.ErrClosed`; - the bytes that were still buffered are discarded with it. A read that is already blocked in another goroutine unblocks with the same error. Nothing panics. If you were doing `io.ReadAll` and ignoring or mishandling the error, you keep a short prefix of the output and carry on — which is why this bug is usually reported as "the command's output is sometimes truncated", and why it is size-dependent and looks flaky. Small outputs fit entirely in the buffer and are read before `Wait` runs, so the bug hides until the data grows. ## The correct sequence ```go stdout, err := cmd.StdoutPipe() if err != nil { return err } if err := cmd.Start(); err != nil { return err } data, err := io.ReadAll(stdout) // finish the reads first if err != nil { return err } if err := cmd.Wait(); err != nil { // now Wait may close the pipe return err } ``` Read until end of file, *then* `Wait`. End of file on that pipe arrives when the child's write end is closed, which happens when the child exits — so this ordering is not a race you can lose, it is simply the direction the dependency runs. Note also what you must **not** do: calling a helper that combines start and wait in one call and then reading the pipe afterwards is exactly the same mistake compressed into a single line. ## Two pipes need concurrency, not sequence If you take both `cmd.StdoutPipe()` and `cmd.StderrPipe()`, reading them one after the other can hang rather than truncate. While you drain standard output, the child keeps writing diagnostics; the standard-error pipe is small and fills; the child blocks on its next write to standard error and therefore never finishes writing standard output; your read never reaches end of file. Drain both on separate goroutines, or take only one pipe and assign a writer to the other stream so os/exec drains it for you. ## When Wait blocks even though the child is gone There is a mirror-image surprise. Because `Wait` also waits for the pipe I/O to finish, it can block long after the process has exited — typically when the child spawned a grandchild that inherited the write end of the pipe. The pipe stays open, the copy never sees end of file, and `Wait` sits there. `cmd.WaitDelay` bounds that. Set it to a non-zero duration and, once the process has exited and the delay has elapsed with the I/O still outstanding, `Wait` closes the pipes itself and reports `exec.ErrWaitDelay` alongside whatever else it has. It is the difference between a job that finishes with a diagnosable error and a job that hangs a build agent overnight. ## The way to avoid the whole class Every hazard above comes from owning the pipe. Assigning a writer instead — `cmd.Stdout = &buf`, or any writer of your choosing — puts the draining inside os/exec, and `cmd.Wait` joins that copy before returning, so the output is complete exactly when `Wait` returns and no ordering rule exists for you to break. Take the pipe only when you need to act on the bytes while the child is still running, and when you do, treat "read to end of file, then `Wait`" as one indivisible step that nothing may come between.

  • You take both cmd.StdoutPipe() and cmd.StderrPipe(). What is the extra hazard?
    Draining them one after the other can hang. While you read standard output, the child fills the standard-error pipe, blocks on its next diagnostic write and never finishes its output, so your read never reaches end of file. Drain both on separate goroutines, or assign a writer to one stream so os/exec drains it.
  • cmd.Wait blocks even though the child has already exited. What is happening, and what bounds it?
    Wait also waits for the pipe I/O to complete, and a grandchild that inherited the write end keeps the pipe open, so the copy never sees end of file. A non-zero cmd.WaitDelay bounds it: after the process exits and the delay passes, Wait closes the pipes and reports exec.ErrWaitDelay instead of hanging.
  • Do you need to Close the reader returned by cmd.StdoutPipe yourself?
    Normally no — cmd.Wait closes it after seeing the command exit, which is why the documentation says most callers need not close it. Closing it early is a deliberate act: it stops the child's writes from going anywhere and can make the child fail on a broken pipe, so do it only when you mean to abandon the output.

saying these in an interview costs you the question

  • Calls cmd.Wait first and then reads the stdout pipe
  • Thinks the kernel holds output until somebody reads it
  • Blames the child for truncating its own output
  • Drains stdout and stderr pipes sequentially
  • Says skipping cmd.Wait avoids the problem