skip to content

A bash cleanup script uses `for f in $(ls *.log); do rm -- "$f"; done` and fails with "No such file or directory" on a file named `app server.log`. Why does the loop break, and what is the correct way to iterate over those files?

level: juniorimportance: should knowfreq 60%

answer

  1. count the words, not the files
  2. who splits the ls output?
  3. IFS splits on the space
  4. a glob is already a list
  5. drop ls, iterate the pattern

basics

~20 s

Command substitution output is split on whitespace, so a filename containing a space becomes two loop items. Drop ls entirely and let the glob supply the list: for f in *.log, which yields one item per file no matter what the names contain.

solid answer

~50 s

`$(ls *.log)` produces one big string, and bash then word-splits it on the characters in `IFS` — space, tab and newline. `app server.log` is therefore split into `app` and `server.log`, and the loop runs `rm` on two names that do not exist. The unquoted result is also subject to pathname expansion, so a filename containing `*` or `?` can expand again into something else entirely. The fix is to skip `ls` and iterate the glob directly: `for f in *.log; do rm -- "$f"; done`. Bash expands a glob into a list of separate words *before* the loop starts, and a matched filename is never re-split, whatever it contains. Quote `"$f"` in the body and pass `--` so a name starting with a dash is not read as an option. If nothing matches, the glob stays literal unless you set `shopt -s nullglob`.

code

bash · 8 lines
bash
mkdir -p /tmp/globdemo && cd /tmp/globdemo
touch 'app server.log' other.log

echo '--- parsing ls ---'
for f in $(ls *.log); do echo "[$f]"; done

echo '--- iterating the glob ---'
for f in *.log; do echo "[$f]"; done

go deeper

for a junior

Know the rule outright: never loop over ls output, loop over the glob. Be able to say that the space in a filename splits the substitution into two words, and always quote "$f" in the body.

for a middle

Explain the mechanism — command substitution is split on IFS and then pathname-expanded, while a glob produces separate words that are never re-split — and cover the no-match case with nullglob or failglob.

for a senior

Extend it to real inputs: user-supplied filenames with newlines or leading dashes, recursing with globstar versus find -print0 into a read -d '' loop, and why -- and quoting defend against different failures.

for a principal

Treat it as a lint-enforceable class rather than a code-review catch: ShellCheck's SC2045/SC2086 in CI, a house rule that file lists come from globs or NUL-delimited find, and awareness that these bugs surface only on data you did not test with.

## What the loop actually receives `for` iterates over a list of *words*. Everything hinges on how that list is produced. With `for f in $(ls *.log)`, bash runs `ls`, captures its output as a single string such as `app server.log\nother.log\n`, and then — because the substitution is unquoted — splits that string on the characters in `IFS` (space, tab, newline by default). The list becomes three words: `app`, `server.log`, `other.log`. The loop dutifully runs three iterations, and two of them name files that do not exist. There is a second hazard in the same construct: the split words then undergo pathname expansion. A file literally named `*` would expand into every name in the directory. Quoting `"$f"` in the body, which the script already does, protects the `rm` call but cannot undo damage done while the list was being built. ## The fix: let the glob be the list ```bash for f in *.log; do rm -- "$f" done ``` Pathname expansion produces a list of words *directly*. Each matching filename is one word from the start, and results of an expansion are not re-scanned for further splitting, so `app server.log` stays a single item. No external command runs, so there is no output to parse and nothing to mis-parse. This is why the rule of thumb is "never parse `ls`" — ShellCheck raises `SC2045` on `for f in $(ls ...)` for exactly this reason. Two details in the body matter: - Quote `"$f"`. Unquoted, the same word splitting happens all over again at the point of use. - Pass `--` before the filename. A file called `-rf` or `--help` would otherwise be parsed as options by `rm`; `--` tells the command that everything after it is an operand. ## What happens when nothing matches By default a glob that matches nothing is left alone as literal text, so `for f in *.log` runs *one* iteration with `f` set to the string `*.log`, and `rm -- "*.log"` fails. Handle it in one of two ways: ```bash shopt -s nullglob # non-matching glob expands to nothing: zero iterations for f in *.log; do rm -- "$f"; done ``` or guard inside the body with `[[ -e $f ]] || continue`. There is also `shopt -s failglob`, which makes a non-matching glob an error instead — useful in a strict script where a missing file means the assumptions are wrong. Note that `nullglob` is global once set, so setting it changes every subsequent glob in the script. ## Recursion and other directories `*.log` matches only the current directory, and only names not starting with a dot. For a whole tree you have two idiomatic choices: ```bash shopt -s globstar # bash 4.0+ for f in ./**/*.log; do rm -- "$f"; done ``` or drive it from `find`, reading NUL-separated names so that even a filename containing a newline survives: ```bash while IFS= read -r -d '' f; do rm -- "$f" done < <(find . -name '*.log' -print0) ``` The `< <( ... )` shape rather than a pipe matters here for the same reason it always does: it keeps the loop in the current shell so anything the body assigns is still there afterwards. ## Why `for` over lines is not a substitute People sometimes "fix" this with `IFS=$'\n'` so that only newlines split the `ls` output. It is better, but it is still parsing `ls`: it breaks on filenames containing newlines (legal on Linux), it depends on a global `IFS` change that affects the rest of the script, and it still runs the unwanted extra pathname expansion. The glob and the `find -print0` loop have neither problem, and both are shorter.

  • What does the loop do when no file matches `*.log`?
    By default the glob is left as literal text, so the loop runs once with `f` set to the string `*.log` and the body fails on a file that does not exist. Either set `shopt -s nullglob` so a non-matching glob expands to nothing, or guard the body with `[[ -e $f ]] || continue`.
  • Why pass `--` to rm inside the loop?
    A filename can begin with a dash, and `rm -rf` reads that as options rather than as an operand. `--` marks the end of option parsing, so `rm -- "$f"` treats a file called `-rf` or `--help` as a name. Combine it with quoting; the two protect against different things.

saying these in an interview costs you the question

  • Says quoting $(ls *.log) in the for list fixes it
  • Blames rm rather than how the list was built
  • Claims ls escapes names with spaces for you
  • Thinks a glob output needs re-quoting after expansion
  • Sets IFS globally to newline and calls it solved

context