A backup script runs `pg_dump mydb > >(gzip -9 > dump.sql.gz)`, reports success, and yet the .gz file is occasionally truncated while a gzip failure never fails the script. What is bash doing with `>(gzip …)`, and why is that process's exit status invisible?

## What `> >(cmd)` actually sets up Again, two constructs side by side. `>(gzip -9 > dump.sql.gz)` is process substitution in the write direction: bash creates a pipe, forks a process running `gzip` with its **stdin** on the read end, and expands the word to a path — `/dev/fd/63` on Linux — naming the write end. The outer `>` is then ordinary output redirection, pointing `pg_dump`'s stdout at that path. So the data flow is the same as a pipeline. The *bookkeeping* is not, and that is where both symptoms come from. ## Why `$?` says nothing about gzip Bash tracks exit statuses for the things it waits on: the foreground command, the stages of a pipeline (individually, in `PIPESTATUS`), and background jobs you `wait` for. A process substitution is none of those from the outer command's point of view — it is a *filename*. After the line runs, `$?` holds `pg_dump`'s status and nothing else. This is worth stating plainly because the usual defences do not apply: - `set -e` sees a zero status and does not fire. - `set -o pipefail` only affects pipelines, and there is no pipeline on this line. - `${PIPESTATUS[@]}` has a single element, again `pg_dump`'s. So `gzip` exiting non-zero because the filesystem filled up produces a green script and a corrupt artifact — the worst possible combination, because the backup is now believed good. ## The truncation: nobody waits When `pg_dump` finishes, its end of the pipe closes and bash proceeds to the next statement. `gzip` is still draining its buffer and writing the tail of the archive. Nothing in the language forces the shell to wait for it. If the next thing the script does is exit — and a backup script's last real line usually is the dump — then whatever supervises the script may reap the process group immediately: a container entrypoint exits, a CI job tears down, a systemd unit with a default `KillMode` stops the cgroup. `gzip` dies mid-flush and you get a `.gz` that fails `gzip -t`. Bash's reaping of process substitutions has varied across releases and is not something to build on. Treat "the writer has finished when the next line runs" as false and design accordingly. ## What to do instead **Use a real pipeline.** For a single consumer there is no reason to reach for `>( )` at all: ```bash set -o pipefail pg_dump mydb | gzip -9 > dump.sql.gz ``` Both processes are pipeline stages, the shell waits for both, `pipefail` surfaces either failure, and `${PIPESTATUS[@]}` tells you which one broke. **Use an explicit FIFO when you truly need a filename.** When a tool must be handed a path and its failure matters, build the plumbing yourself so there is a PID to wait on: ```bash mkfifo "$dir/pipe" gzip -9 < "$dir/pipe" > dump.sql.gz & writer=$! pg_dump mydb > "$dir/pipe" wait "$writer" || die 'compression failed' ``` **Or write, then verify.** Dump to a temporary file, compress it, check the status, and only then move it into place. Slower and it needs the disk, but the artifact is either correct or absent — never a truncated file that looks finished. ## When `>( )` is still the right tool Fan-out is the genuine use case, because no single pipeline can feed two consumers: ```bash build_output | tee >(gzip -9 > log.gz) >(grep -c ERROR > errors.txt) > /dev/null ``` That is fine for observability side-channels whose failure you are willing to ignore. It stops being fine the moment one of those branches produces the artifact you will restore from. The rule of thumb: process substitution is a convenience for *arranging streams*, not a mechanism for *reliable delivery*. If a downstream failure must fail the script, put that consumer somewhere the shell will wait for it and report on it. ## Diagnosing it in the wild The signature is a script that always exits 0, an output file whose size varies between runs, and `gzip -t` or `tar -tzf` failing on the artifact. Adding `-o pipefail` changes nothing — which itself is the clue that there is no pipeline. Reproduce it by making the writer slow (`gzip -9` on a large dump) and having the script exit immediately after; the race widens until it is deterministic.