skip to content

In bash, why is `files=($(find . -name '*.log'))` an unsafe way to capture a list of filenames into an array, and what would you use instead?

level: seniorimportance: should knowfreq 40%

answer

  1. a substitution is text, not a list
  2. splitting and globbing both apply
  3. one element per line, not per word
  4. NUL is the only safe delimiter

basics

~20 s

The unquoted substitution is split on IFS and then glob-expanded, so any filename containing a space, tab or newline becomes several elements and one containing a glob character can be replaced by other names. Fill the array with mapfile -t, or a NUL-delimited read loop.

solid answer

~50 s

An unquoted command substitution is not a list — it is one blob of text that bash then splits on `IFS` and runs pathname expansion over. A file called `app error.log` becomes two elements, a file whose name contains a newline becomes two more, and an element that happens to contain `*` or `[` can be replaced by whatever matches in the current directory. Element count and contents are therefore attacker- and user-influenced. For line-oriented data use `mapfile -t files < <(find . -name '*.log')` (bash 4.0+), which puts one line in one element and `-t` strips the trailing newline. For filenames, which may legally contain newlines, go NUL-delimited: `mapfile -d '' -t files < <(find . -print0)` on bash 4.4+, or a `while IFS= read -r -d '' f; do files+=("$f"); done` loop on older bash such as the 3.2 that ships on macOS.

code

bash · 10 lines
bash
# line-oriented, bash 4.0+: one line per element, newline stripped
mapfile -t hosts < /etc/hosts
printf '%d lines\n' "${#hosts[@]}"

# filename-safe on any bash: NUL-delimited read loop
files=()
while IFS= read -r -d '' f; do
  files+=("$f")
done < <(find . -maxdepth 1 -type f -print0)
printf '[%s]\n' "${files[@]}"

go deeper

for a junior

Know that wrapping a command substitution in parentheses does not create a proper list, and that mapfile -t is the normal way to read lines into an array.

for a middle

Explain both damaging steps — word splitting on IFS and pathname expansion — and show mapfile -t with a redirect from process substitution rather than a pipe.

for a senior

Bring the operational judgment: filenames from real users contain spaces and can contain newlines, so choose NUL delimiting for paths, state the bash version each construct needs, and verify element boundaries with printf before trusting a count.

for a principal

Decide the standard for the codebase: which bash version scripts may assume, whether a 3.2-compatible fallback is required for developer laptops, and at what point a traversal with real filename handling should stop being a shell script at all.

## What goes wrong ```bash files=($(find . -name '*.log')) ``` Command substitution yields a single string containing the command's stdout with trailing newlines removed. Inside the array assignment the words are unquoted, so bash performs two more steps on that string: 1. **Word splitting** on `IFS`, whose default is space, tab and newline. Every one of those characters inside a filename becomes an element boundary. 2. **Pathname expansion** on each resulting word. A word containing `*`, `?` or `[...]` is replaced by matching filenames — or, if nothing matches, left as-is, which is a different kind of surprise. So `./app error.log` becomes the elements `./app` and `error.log`, neither of which exists. A file literally named `*` expands to every entry in the directory. The array's length no longer corresponds to the number of files, and later code that trusts `${#files[@]}` is now wrong. This is not a theoretical concern in production: filenames from uploads, exports, media libraries and Windows-originated shares routinely contain spaces, and newlines in filenames are legal on every POSIX filesystem. ## The line-oriented fix: mapfile ```bash mapfile -t lines < /var/log/app/index.txt ``` `mapfile` (also spelled `readarray`) reads lines from standard input into an array, one line per element, with no splitting and no globbing. `-t` removes the trailing newline from each line, which you almost always want. It arrived in bash 4.0, so it is unavailable on the bash 3.2 that Apple ships. To read from a command rather than a file, redirect from a process substitution: ```bash mapfile -t files < <(find . -name '*.log') ``` The redirection matters. `find ... | mapfile -t files` leaves the array empty in a default bash, because the right-hand side of a pipeline runs in a subshell whose variables die with it — the mechanics of that belong to subshell semantics, but the practical rule is: feed `mapfile` with `< <( ... )`, not with a pipe. ## The filename-safe fix: NUL delimiters A newline is a legal character in a filename, so line-oriented reading is still ambiguous for paths. The only byte that cannot appear in a filename is NUL, which is why `find` offers `-print0`: ```bash mapfile -d '' -t files < <(find . -name '*.log' -print0) # bash 4.4+ ``` `-d ''` sets the delimiter to NUL. It was added in bash 4.4. On anything older, or when you want maximum portability, use the read loop, which works back to bash 3.2: ```bash files=() while IFS= read -r -d '' f; do files+=("$f") done < <(find . -name '*.log' -print0) ``` Each appended value is quoted, so it stays a single element regardless of its contents. ## Checking what you got After filling an array, the cheap sanity check is to print boundaries rather than a joined string: ```bash printf '%d files\n' "${#files[@]}" printf '[%s]\n' "${files[@]}" ``` If a single file has become two elements, this shows it immediately, whereas `echo ${files[@]}` hides every boundary. One more detail: when a directory has no matches, `find` prints nothing, so `mapfile` produces an empty array and `${#files[@]}` is 0. That is a clean, testable outcome — better than the glob-based alternative, where an unmatched pattern is left literal unless `nullglob` is set. ## When globbing is the better tool If you only need files in known directories and no recursion, a plain glob is both simpler and inherently safe, because pathname expansion produces a proper word list without passing through text: ```bash files=(./logs/*.log) ``` Each match is one element, spaces and all, with no splitting involved. Reach for `find` plus `mapfile` when you need recursion, predicates such as `-mtime`, or a traversal you cannot express as a pattern. ## What an interviewer is listening for The weak answer is "quote it" — `files=("$(find ...)")` is worse, since it produces exactly one element containing the whole output. The strong answer names the two expansion steps that damage the data, distinguishes line-safe from filename-safe, and states the bash version each solution needs.

  • Would `files=("$(find . -name '*.log')")` fix it?
    No — it makes it worse in a different way. Quoting the substitution suppresses splitting entirely, so the whole multi-line output becomes a single element. You then have one array entry containing every path separated by newlines, and `${#files[@]}` is 1 no matter how many files matched.
  • Why does `find . -print0 | mapfile -d '' -t files` leave the array empty?
    The right-hand side of a pipeline runs in a subshell, so mapfile fills an array that is discarded when that subshell exits. Redirect from a process substitution instead — `mapfile -d '' -t files < <(find . -print0)` — so mapfile runs in the current shell. The `lastpipe` shopt is an alternative, but it only applies with job control off.
  • When is a plain glob preferable to find plus mapfile?
    Whenever the set of files is expressible as a pattern in known directories. `files=(./logs/*.log)` produces one element per match with no text stage in between, so nothing can be re-split. Add `shopt -s nullglob` so an unmatched pattern yields an empty array rather than the literal pattern. Use find when you need recursion or predicates such as -mtime.

saying these in an interview costs you the question

  • Assuming filenames never contain spaces or newlines
  • Quoting the whole command substitution to fix splitting
  • Piping into mapfile and expecting the array to persist
  • Thinking mapfile is available on macOS's default bash
  • Setting IFS to newline and calling the problem solved

context