skip to content

A cleanup script iterates with `for f in $(ls /var/uploads)` and then runs `rm $f`, over a directory where users choose the uploaded filenames. Which filenames break or subvert that loop, and how would you write it correctly?

level: middleimportance: must knowfreq 72%

answer

  1. ls output is text, not a list
  2. IFS decides where names end
  3. a name may contain a newline
  4. a dash in operand position is a flag
  5. globs produce words directly

basics

~20 s

Word splitting and globbing chew the ls output: a name with a space or newline becomes several loop items, one containing an asterisk expands against the directory, and one starting with a dash is read by rm as an option. Loop over a glob and quote.

solid answer

~50 s

The command substitution produces one blob of text, and because it is unquoted bash applies word splitting on IFS and then pathname expansion to it. So `annual report.pdf` becomes two iterations, a filename containing a newline splits into two, and a filename containing `*` or `?` is expanded against the current directory — which can hand `rm` files the user never uploaded. Then `rm $f` splits and globs a second time, and a file called `-rf` or `--help` is parsed by rm as options rather than an operand. On top of that, `ls` formats for humans and is not an interchange format. The correct loop iterates a glob directly: `for f in /var/uploads/*; do [[ -f $f ]] || continue; rm -- "$f"; done`. The glob yields whole pathnames with no splitting, the directory prefix removes the leading-dash problem, and `--` ends option parsing anyway.

code

bash · 9 lines
bash
#!/usr/bin/env bash
set -euo pipefail
shopt -s nullglob

dir=/var/uploads
for f in "$dir"/*; do
  [[ -f $f ]] || continue
  rm -- "$f"        # -- keeps a file named "-rf" out of option position
done

go deeper

for a junior

Know that an unquoted expansion is split into words at spaces, so a file called annual report.pdf becomes two arguments. Always quote expansions and prefer looping over a glob rather than over the output of ls.

for a middle

Walk the three stages — command substitution, word splitting on IFS, pathname expansion — and show which filename breaks each one, including the newline case that makes line-based lists ambiguous. Explain what -- and a ./ prefix buy you.

for a senior

Treat a user-writable directory as adversarial input: expect names containing globs, newlines and leading dashes, and pick a channel that cannot lose the boundary, such as a glob or a NUL-delimited find stream. Be able to review a cleanup script and say what it would delete in the worst case.

for a principal

Set the standard: filename handling is a review checklist item and a lint gate, and scripts that touch untrusted directories carry a dry-run mode and a bounded blast radius. Decide when the file-marshalling logic should move out of shell entirely.

## What the broken loop actually evaluates `for f in $(ls /var/uploads)` runs through three separate mechanisms, and each one is a chance for a filename to stop behaving like a filename. 1. **Command substitution** captures ls's standard output as one string and strips trailing newlines. 2. **Word splitting** then cuts that string at every character in IFS — by default space, tab and newline. Nothing about a filename tells bash where one ends; only IFS does. 3. **Pathname expansion** runs on each resulting word, so any word containing `*`, `?` or `[...]` is replaced by whatever matches in the *current* directory. Then the body repeats steps 2 and 3, because `rm $f` is unquoted too. ## The four filename shapes that break it **A space or tab.** `annual report.pdf` becomes two words. The loop runs `rm annual` and `rm report.pdf`, both of which fail — or, worse, succeed on unrelated files that happen to have those names. **A newline.** Filenames may contain any byte except NUL and `/`, newline included. One file becomes two iterations, and no amount of reading the list line by line can tell the two cases apart. This is the case that makes newline-delimited filename lists fundamentally ambiguous. **A glob character.** A user uploads a file literally named `*`. After splitting, that word undergoes pathname expansion and becomes *every* name in the current directory. A cleanup script can be made to delete files it was never pointed at. **A leading dash.** A file named `-rf`, `-f` or `--help` is passed to rm in option position. `rm` has no way to know it came from a variable; it reads it as flags. This is how a delete-one-file loop turns into a recursive force delete. ## Why parsing ls is the wrong starting point `ls` is a display tool. Its output layout depends on whether stdout is a terminal, on flags, and on the implementation; GNU coreutils will even quote or replace unprintable characters in some modes. There is no escaping convention you can reliably reverse. The shell already has a filename-listing primitive that returns *words, not text*: the glob. ## The correct idioms **Iterate a glob.** ```bash shopt -s nullglob # empty directory: zero iterations, not one literal '*' for f in /var/uploads/*; do [[ -f $f ]] || continue rm -- "$f" done ``` A glob expands to a list of *separate words* directly — no splitting stage runs over it, so spaces and newlines inside a name are harmless. Each word is a full path beginning with `/var/uploads/`, so no name can start with a dash. Without `nullglob`, an unmatched pattern stays literal and the loop runs once with the pattern itself, which is why the `-f` guard (or `nullglob`) is not optional. For a glob in the current directory, write `./*` for the same leading-dash protection. **Quote every expansion, and end option parsing.** `rm -- "$f"` is two defences: the quotes stop re-splitting and re-globbing, and `--` tells rm that everything after it is an operand. Most GNU and BSD tools honour `--`; `./` prefixes cover the ones that do not. **When you need a whole tree, use a NUL-delimited stream.** ```bash while IFS= read -r -d '' f; do rm -- "$f" done < <(find /var/uploads -type f -name '*.tmp' -print0) ``` `-print0` emits NUL-terminated names, `read -d ''` reads up to a NUL, `IFS=` stops leading and trailing whitespace being trimmed, and `-r` stops backslashes being interpreted. Feeding it with process substitution rather than a pipe also keeps the loop in the current shell, which matters if the body sets variables — the reason a piped loop behaves differently is a subshell topic of its own. ## The invariant to state in the interview A filename is a byte string, not a line of text. Any design that moves filenames through a whitespace- or newline-delimited channel is lossy for inputs an attacker chooses. Prefer channels that never lose the boundary: a glob, `find -exec`, or a NUL-delimited stream. And in a directory that untrusted users write to, assume every hostile name — leading dash, embedded newline, embedded glob — is present, because producing one costs nothing.

  • Would setting IFS to a newline before the loop make parsing the ls output safe?
    No. It fixes the space and tab cases but not the two that matter most: a filename may legitimately contain a newline, so one file still becomes two iterations, and pathname expansion still runs on each word unless you also disable globbing with `set -f`. The channel itself is lossy — switch to a glob or a NUL-delimited find stream instead of patching the delimiter.
  • What happens to that glob loop when the directory is empty?
    By default the pattern matches nothing and is left literal, so the loop runs once with `f` set to `/var/uploads/*` and the body operates on a path that does not exist. Either enable `shopt -s nullglob` so an unmatched pattern expands to zero words, or guard the body with a `[[ -e $f ]]` test — most scripts do both.
  • Does -- protect you with every command?
    Not universally. `--` is a convention that most GNU and BSD utilities and all shell builtins that take operands honour, but a few tools and many in-house scripts do not implement it. The portable belt-and-braces move is to make the path itself unambiguous — pass `./name` or an absolute path — so no argument ever begins with a dash.

saying these in an interview costs you the question

  • Nobody puts spaces or newlines in filenames
  • Wrapping the substitution in quotes fixes the loop
  • ls -1 makes the output safe to parse
  • Setting IFS to newline handles every filename
  • rm can tell a variable from a real option

context