skip to content

When the context.Context passed to exec.CommandContext is cancelled, what happens to the child process, and why can Wait still hang?

level: seniorimportance: nice to knowfreq 32%

answer

  1. Cancel is a field, and it defaults to Kill
  2. SIGKILL, so no flush and no cleanup
  3. Wait also waits on the copying goroutines
  4. A grandchild still holds the pipe open
  5. WaitDelay turns a hang into an error

basics

~20 s

By default the child is killed outright: exec.CommandContext installs a Cancel func that calls Process.Kill, which is SIGKILL and ungraceful. Wait can still hang afterwards, because it also waits for the goroutines copying the inherited pipes.

solid answer

~40 s

`exec.CommandContext` wires the context to the process by installing a `Cmd.Cancel` function that calls `Process.Kill` when Done closes - a SIGKILL, so the child gets no chance to flush or clean up. But killing the child need not end `Wait`. If you set `cmd.Stdout` or `cmd.Stderr` to an `io.Writer` rather than an `*os.File`, `os/exec` creates a pipe and a copying goroutine, and `Wait` blocks until that copy sees EOF - which happens only when every holder of the write end is gone. A grandchild the command spawned inherited that descriptor, so `Wait` waits forever on a process that is already dead. The fixes: set `Cmd.Cancel` yourself to send a gentler signal, and set `Cmd.WaitDelay`, which bounds how long `Wait` waits after cancellation or exit before killing the process, closing the pipes and returning `exec.ErrWaitDelay`.

code

go · 10 lines
go
cmd := exec.CommandContext(ctx, "tar", "-czf", dst, src)
cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) }
cmd.WaitDelay = 5 * time.Second // then kill and close the pipes

var out bytes.Buffer
cmd.Stdout = &out // a non-*os.File writer, so os/exec makes a pipe

if err := cmd.Run(); err != nil {
	return fmt.Errorf("archive %s: %w", src, err)
}

go deeper

for a junior

Know that exec.CommandContext ties the child's lifetime to the context and that the default action on cancellation is to kill the process outright, with no graceful shutdown.

for a middle

Explain that Cmd.Cancel is a settable field defaulting to Process.Kill, and that Wait covers both the process and the goroutines copying any non-file Stdout or Stderr.

for a senior

Diagnose the hang: process reaped, Wait still parked, a grandchild holding the pipe. Reach for WaitDelay as the bound and a custom Cancel as the courtesy, and say what partial output the kill can leave behind.

for a principal

Decide the shutdown contract for anything the platform shells out to: how long a child gets to finish, whether partial artefacts are ever allowed to be visible, and whether commands must run in their own process group so a cancel stops the whole tree.

## What CommandContext actually wires up `exec.CommandContext(ctx, name, args...)` builds the same `*exec.Cmd` as `exec.Command`, and additionally sets the command's context and a default `Cmd.Cancel` function equivalent to `func() error { return cmd.Process.Kill() }`. When `ctx.Done()` closes while the command is running, `os/exec` calls `Cancel`. `Process.Kill` is SIGKILL on Unix: unblockable, uncatchable, no chance for the child to flush a partially written output file, remove a temp file, or commit anything in progress. That is often wrong for a batch job that shells out to a transform step. The child may be halfway through writing an output file, and SIGKILL leaves it truncated with no marker saying so. Setting `cmd.Cancel` to send `os.Interrupt` (or SIGTERM) instead gives well-behaved children a chance to exit cleanly - as long as you also bound how long you are willing to wait for that. ## The part that surprises people: Wait is not just waitpid `Cmd.Wait` does two things: it reaps the process, and it waits for the I/O goroutines `os/exec` started on your behalf. Those goroutines exist whenever `Stdin`, `Stdout` or `Stderr` is set to something that is not an `*os.File` - a `bytes.Buffer`, a `strings.Reader`, a log writer. For each one, `os/exec` makes an OS pipe, gives one end to the child, and runs a goroutine copying between your writer and the other end. The copy ends when the read side sees EOF, and a pipe reports EOF only when *every* descriptor for its write end is closed. The child had one. If the child spawned a grandchild - a shell script that backgrounds a daemon, a wrapper that forks a helper - that grandchild inherited the same descriptor. Kill the child and the grandchild still holds the write end open, so the copy never finishes, and `Wait` blocks indefinitely on a process that has already been reaped. From the outside the job looks stuck in cancellation, which is the worst possible time for it to be stuck. ## WaitDelay is the bound `Cmd.WaitDelay` is a `time.Duration` field that bounds this. Set it, and after the process exits or the context is cancelled, `os/exec` gives the remaining I/O that long to finish; if it does not, it kills the process if it is still alive, closes the pipes so the copying goroutines unblock, and `Wait` returns `exec.ErrWaitDelay` (unless the command already produced a more interesting error). It converts "hangs forever" into "returns an error after N seconds", which is what an operator needs. Paired with a custom `Cancel`, the shape is: `Cancel` sends SIGTERM so a cooperative child can exit cleanly, and `WaitDelay` guarantees you stop caring within a few seconds either way. ## Other ways this bites - **Using `StdoutPipe` and forgetting the ordering.** `Cmd.StdoutPipe` gives you the read end directly; you must finish reading before calling `Wait`, because `Wait` closes the pipe. Read after `Wait` and you get a closed-pipe error rather than the output. - **Assuming cancellation cleans up the child's side effects.** It does not. Whatever the process wrote stays written. If partial output is not safe, have the child write to a temp path and rename on success, and delete the temp path in your own cleanup. - **Process groups.** SIGKILL to the child does not reach its descendants; they are reparented and keep running. If the command routinely spawns children, launching it in its own process group and signalling the group is the only reliable way to stop the whole tree, and it is worth saying so in an interview even without writing the syscall attributes out. ## How to answer this in a loop State the default (SIGKILL via `Cancel`), state the trap (`Wait` also waits on inherited pipes, so a grandchild can pin it open), and state the two knobs (`Cancel` for how you ask, `WaitDelay` for how long you wait). Then add the operator's view: a cancel that can hang is not a cancel, so the bound is the non-negotiable part and the gentle signal is the nicety.

  • How do you make cancellation send SIGTERM instead of SIGKILL?
    Set `cmd.Cancel` on a command created by `exec.CommandContext`: `cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) }` (or SIGTERM). Always pair it with a `cmd.WaitDelay`, because a child that ignores the polite signal must still be stopped; after the delay os/exec kills it and closes the pipes.
  • Does killing the child also stop the processes it spawned?
    No. A signal to one process does not reach its descendants; they are reparented and keep running, and they may still hold the inherited pipe write end. Stopping a whole tree means launching the command in its own process group and signalling the group.
  • Why does redirecting output to an *os.File avoid the Wait hang?
    Because os/exec passes an `*os.File` straight to the child as its descriptor with no pipe and no copying goroutine. There is nothing for `Wait` to wait on beyond the process itself, so a killed child ends `Wait` immediately.

saying these in an interview costs you the question

  • Says cancellation sends SIGTERM so the child can clean up
  • Assumes Wait returns as soon as the process dies
  • Thinks killing the child also kills its descendants
  • Believes cancellation undoes what the child already wrote
  • Reads from StdoutPipe after calling Wait