What can go wrong when shutil.rmtree deletes a directory tree?

## A destructive call deserves the same scrutiny as a database delete `shutil.rmtree(path)` removes a directory and everything under it. It is the counterpart to `os.rmdir`, which only works on an empty directory, and it is the engine behind `tempfile.TemporaryDirectory` cleanup. Because it destroys data, its failure modes are worth knowing in detail rather than discovering during an incident. ## It is a walk, not an operation The first thing to internalise is that `rmtree` is not atomic. It descends the tree unlinking entries and removing directories as it goes. If it raises halfway — one unreadable subdirectory, one file another process still holds open — everything it already removed stays removed. A nightly cleanup that deletes a payment reconciliation job's staging root and fails on the third of four run directories has not "failed safely"; it has left the tree in a state no code expected, and a retry now sees a half-empty directory. Where the intermediate state is visible to other processes, the standard trick is to make the disappearance atomic and the deletion lazy: rename the tree to a scratch name inside the same directory with `os.rename`, which is instantaneous and atomic within a filesystem, and then `rmtree` the renamed path. Observers see the directory vanish at once, and a failure during the slow removal leaves only orphaned scratch, not a partially gutted live tree. ## Error handling: `onexc`, `onerror` and `ignore_errors` By default any failure propagates. Two knobs change that. `ignore_errors=True` discards every error and keeps going, leaving whatever could not be removed. It is right for genuinely best-effort cleanup and wrong anywhere you need to know the directory is actually gone, because it turns a failure into silence. `onexc=handler` is the precise tool, added in Python 3.12. The handler is called as `handler(function, path, exception)` — the operation that failed (`os.unlink`, `os.rmdir`, `os.scandir`), the path it failed on, and the exception *instance*. It can inspect, log, repair and retry, or re-raise to abort the walk. The older `onerror` parameter is the same idea with a worse signature: it receives the three-tuple from `sys.exc_info()` instead of the exception. Python 3.12 documented `onerror` as deprecated in favour of `onexc`; it still functions on Python 3.14, so old code keeps working, but new code should use `onexc`. The canonical handler clears a read-only bit and retries. On Windows a read-only file genuinely cannot be deleted, which is why unguarded `rmtree` in cross-platform cleanup code fails there and passes on Unix — on Unix, deletion depends on the *directory's* write permission, not the file's, so the same tree removes cleanly and the bug hides until CI runs on another platform. ## Symlinks, in three distinct cases These get conflated constantly, and they are three different behaviours. 1. **The root is a symlink to a directory.** `rmtree` refuses and raises `OSError`. It will not delete the linked-to tree. Use `os.unlink` if what you wanted was to remove the link. 2. **A symlink is found inside the tree.** It is unlinked like any other entry; the target is not followed and its contents survive. So `rmtree` cannot be lured out of the subtree by a link someone left in it. 3. **A symlink is swapped in during the walk.** This is the hostile case: an attacker who can create entries in the tree replaces a directory with a link between the moment `rmtree` inspects it and the moment it recurses. On platforms that provide the descriptor-relative directory calls, `rmtree` uses an implementation that opens each directory and operates relative to that descriptor, which defeats the swap. The module publishes a boolean flag stating whether the running platform gets that guarantee, and where it is false, `rmtree` on a directory writable by anyone else is genuinely unsafe. ## Other things that actually break it - **Open handles.** On Windows a file another process (an editor, an indexer, a virus scanner) has open cannot be deleted. This is the main reason `tempfile.TemporaryDirectory` grew `ignore_cleanup_errors=True` in Python 3.10. - **Network filesystems.** Some NFS servers rename a still-open file to a hidden entry instead of deleting it, so the parent `os.rmdir` then fails with a not-empty error even though your code deleted everything it saw. - **The path itself.** `rmtree` on a plain file raises rather than unlinking it, and on a missing path raises `FileNotFoundError` — there is no `missing_ok` equivalent, so either catch it or pass `ignore_errors=True`. - **Unvalidated input.** This is the one that ends careers. A path assembled from a request parameter, a config value or an environment variable, then handed to `rmtree`, is arbitrary deletion. Resolve the path, confirm it is inside the root you own, and refuse otherwise — and prefer deleting a directory your own code created and remembered over one whose name arrived from outside. ## What a good answer sounds like Name the non-atomicity and the rename-then-remove pattern, distinguish `ignore_errors` from `onexc`, give the read-only retry as the concrete handler, and separate the three symlink cases. That combination shows someone who has actually cleaned up after a long-running job rather than someone who has only read the signature.