skip to content

A cleanup step runs `rm -rf "$dir"/*` but `.env` and `.cache` survive inside $dir, so a colleague adds `rm -rf "$dir"/.*`. Explain both behaviours in terms of bash pathname expansion, and give a safe fix.

level: seniorimportance: should knowfreq 44%

answer

  1. a leading dot must be matched explicitly
  2. hidden is a shell convention, not a filesystem one
  3. dot and dot-dot are real entries
  4. the parent directory ends up on the command line
  5. name the directory, not its contents

basics

~20 s

A leading dot must be matched explicitly, so * never expands to hidden names and they survive the cleanup. The proposed .* is worse: it also matches . and .., putting the parent directory on the command line. Use shopt -s dotglob, or remove and recreate the directory.

solid answer

~50 s

Bash's pathname expansion treats a dot at the start of a name as significant: it is matched only by a pattern that itself begins with a dot. That is why `"$dir"/*` quietly skips `.env`, `.cache` and `.git`, leaving a directory you believed was empty. The `.*` "fix" is a classic foot-gun, because `.` and `..` are real directory entries and both match, so `rm -rf` receives the parent directory as an argument. GNU rm refuses arguments ending in `.` or `..` with a "refusing to remove" message, but that is a safety net in coreutils rather than a guarantee across implementations or across other tools, and you should never rely on it. The clean fixes are: enable `shopt -s dotglob` so `*` includes hidden names, or sidestep globbing entirely by removing and recreating the directory, or by using `find "$dir" -mindepth 1 -delete`. Note that dotglob is shell state, so scope it deliberately rather than flipping it globally in a long script.

code

bash · 10 lines
bash
d=$(mktemp -d); touch "$d/visible" "$d/.env"

rm -rf -- "$d"/*
ls -A "$d"                 # .env survived

( shopt -s dotglob; rm -rf -- "$d"/*; )   # scoped to a subshell
ls -A "$d"                 # now empty

# what the dangerous "fix" actually expands to:
printf '%s\n' "$d"/.*      # includes "$d/." and "$d/.."

go deeper

for a junior

Remember that * never matches names beginning with a dot, so hidden files survive a rm -rf dir/*. Know that ls -a is how you check what was left behind.

for a middle

Explain the explicit-match rule and why .* also expands to . and .., and name dotglob as the option that changes the first behaviour but not the second.

for a senior

Show production judgment: prefer removing and recreating the directory or using find over enabling a global shell option, and reason about the blast radius when the expansion is fed to chmod or chown rather than to GNU rm.

for a principal

Treat this as a review rule rather than a trick: destructive steps should name a path they own instead of enumerating a directory's contents, and pipelines should verify the post-state rather than trusting a cleanup command to have meant what it looked like.

## The rule behind both bugs The bash manual states it plainly: when a pattern is used for pathname expansion, a `.` at the start of a name, or immediately following a slash, must be matched explicitly unless the `dotglob` shell option is set. There is no filesystem-level concept of a hidden file on Unix — "hidden" is entirely a convention implemented by shells and by `ls`, and this expansion rule is where the shell implements it. So: ```bash d=$(mktemp -d); touch "$d/a" "$d/.env" rm -rf "$d"/* ls -A "$d" # .env ``` The cleanup looks like it worked. Every visible file is gone, and only `ls -A` or `ls -a` reveals what stayed. In a build pipeline this manifests as stale credentials, a stale `.cache`, or a `.git` directory shipped into an artefact — failures that appear one run later and look unrelated to the cleanup step. ## Why `.*` is worse than the bug it fixes Every directory contains the entries `.` (itself) and `..` (its parent). Both begin with a dot, and the pattern `.*` begins with a dot, so both match: ```bash echo "$d"/.* # /tmp/xxx/. /tmp/xxx/.. /tmp/xxx/.env ``` Handing that to `rm -rf` means asking to recursively delete the parent directory. GNU coreutils has protected against exactly this for many years: `rm` refuses any argument whose final component is `.` or `..`, printing "refusing to remove '.' or '..' directory: skipping". That protection saves most people most of the time, but it is a property of one implementation of one command. Other tools given the same expansion — `chown -R`, `chmod -R`, `cp -r`, `tar`, a `for` loop calling something custom — have no such rule, and `chmod -R 777 "$dir"/.*` will happily walk up and out. There is a related detail worth knowing: setting `GLOBIGNORE` to a non-null value causes `.` and `..` to be ignored by pathname expansion, and also has the side effect of enabling dotglob. That is a real mechanism, but it is obscure enough that reaching for it in a script trades one surprise for another. The hand-rolled pattern people use instead is `.[!.]* ..?*`, which matches dot-names other than `.` and `..` in two pieces. It is correct and it is unreadable, and either half can fail to match and be left literal. ## The three fixes that are actually good **Do not glob at all.** If the goal is an empty directory, delete it and recreate it: ```bash rm -rf -- "$dir" mkdir -p -- "$dir" ``` This has no hidden-file case, no `..` case, and one obvious failure mode. It changes ownership and mode, so restore them if they matter. **Let find do the walking.** ```bash find "$dir" -mindepth 1 -delete ``` `find` has no dot rule and `-mindepth 1` excludes the directory itself, so this empties the directory including hidden entries while leaving its metadata intact. **Enable dotglob deliberately.** ```bash shopt -s dotglob rm -rf -- "$dir"/* shopt -u dotglob ``` With `dotglob` set, `*` matches names beginning with a dot; bash still requires a pattern to begin with a dot in order to match `.` and `..`, so this does not reintroduce the parent-directory hazard. Because `shopt` is global shell state, keep the window narrow — enabling it at the top of a large script silently changes every other pattern in the file, and any sourced library you call inherits it too. A subshell is a tidy container: `( shopt -s dotglob; rm -rf -- "$dir"/*; )`. ## The copy version of the same bug The symmetrical failure is copying rather than deleting: ```bash cp -r build/* out/ # .env, .github, .dockerignore all missing from out/ ``` Here the idiomatic fix does not involve dotglob at all — use a trailing `/.` so the shell is not doing the selection: ```bash cp -a build/. out/ # copies the directory's contents, hidden entries included rsync -a build/ out/ # trailing slash on the source means the same thing ``` ## The reviewable lesson When a script uses a glob to enumerate everything in a directory, ask whether "everything" was meant literally. If it was, the glob is the wrong tool, because bash's default answer to "everything" excludes an entire class of entries — and in a deployment or cleanup step, that class is disproportionately likely to contain the configuration and credentials you least want left behind. Prefer an operation that names the directory rather than its contents, and reserve `dotglob` for the cases where you really do want the shell to enumerate.

  • Does `shopt -s dotglob` make `*` match `.` and `..` as well?
    No. Bash documents that the names `.` and `..` are matched only by a pattern that itself begins with a dot, even when dotglob is set. That is why dotglob is a safe fix for the cleanup case while writing `.*` by hand is not — the pattern you wrote is what pulls in the parent directory.
  • How do you copy a directory's contents, hidden files included, without touching shell options?
    Use a trailing `/.` on the source so the shell never enumerates anything: `cp -a build/. out/`. The equivalent with rsync is a trailing slash on the source, `rsync -a build/ out/`. Both hand the directory itself to the tool and let it walk the contents, which sidesteps the leading-dot rule entirely.
  • Why is enabling dotglob at the top of a long script risky?
    shopt sets shell state for the rest of the process, including any file you source afterwards. Every other pattern in the script silently starts matching hidden entries, so a later `for f in *` picks up `.git` or editor swap files that the author never considered. Scope it to a subshell or unset it immediately after use.

saying these in an interview costs you the question

  • Assuming `*` matches every entry in a directory
  • Adding `.*` to catch hidden files
  • Believing rm always refuses `..` on every platform
  • Thinking hidden files are a filesystem attribute
  • Enabling dotglob globally and forgetting it is set

context