A Docker host is out of disk mid-build. How do you reclaim space without disrupting running containers?

## Step 1 — find out what is actually full Deleting before measuring is how a cleanup turns into an incident. Two commands, always in this order: ``` docker system df -v sudo du -sh /var/lib/docker/* ``` The first attributes space to images, containers, local volumes and build cache, and itemises each. The second catches everything the daemon does not account for. If `du` says 213 GB and the report only explains 148 GB, the gap is real and it is very often container log files under the data root — a service logging to stdout with no rotation configured on its log driver. That case is fixed by configuring rotation, not by pruning anything, and a candidate who prunes for twenty minutes without noticing is missing the actual bug. Worth knowing at this point: freeing a file that a process still holds open does not return the space. If someone already deleted a huge log out from under the daemon, `df` stays full until the holder closes the descriptor; `lsof +L1` finds those. ## Step 2 — sweep in increasing order of risk Run these one at a time and re-check free space in between, because the first cheap sweep is often enough: 1. **`docker container prune --filter "until=48h"`** — removes stopped containers older than two days along with their writable layers. Running containers are untouchable here by definition. This also *unlocks* their images for a later image sweep, which is why the ordering matters. 2. **`docker image prune`** — dangling images only. Cannot delete anything tagged, cannot delete anything a container references. 3. **`docker builder prune --filter "until=72h"`** — BuildKit cache records not used in three days. On a build host this is frequently the largest single bucket; on a host that only runs containers it is empty. Note it prunes the cache of the current builder; a buildx builder using a container driver holds its cache inside that builder, so you prune it with `docker buildx prune --builder <name>`. 4. **`docker image prune -a --filter "until=168h"`** — the aggressive step, deliberately last and deliberately filtered. It removes every image older than a week that no container references. Before running it, confirm the images you would need for a rollback are pushed and re-pullable, because this is what deletes them. At no point in that ladder can a running container be affected. An image referenced by a running container is never eligible for prune, so an order-checkout API container serving at a 340 ms p99 keeps its image, its writable layer and its volumes throughout. That property is what makes the ladder safe to run during an incident. ## Step 3 — the things that bite **`docker system prune -a --volumes` is not a cleanup, it is a reset.** Plain `docker system prune` already removes stopped containers, networks with nothing attached, dangling images and unused build cache. `-a` extends the image sweep to every image no container uses, and `--volumes` extends it to unused local volumes — that is *data*. On a shared host, the volume of a database container that someone stopped an hour ago is unused by that definition, and it is not recoverable. The rule is simple: on a host you do not exclusively own, do not pass `--volumes`, and prefer the individual prunes so you can see what each one takes. **A running container's writable layer only grows.** If `docker ps -s` shows a container at 47 GB, no prune will help: that space is freed when the container is removed. The fix is to find what it is writing — an application logging to a file path inside the container, a cache directory that was never mounted — and either mount that path as a volume or stop writing there. **Pruning the build cache has a cost you should name out loud.** Deleting BuildKit's cache is completely safe for correctness and directly expensive for build time: the next build of a large Python inference image with CUDA wheels runs cold, re-downloading and re-compiling work that a cache hit would have skipped. During an out-of-disk incident that is the right trade, but on a schedule it is a decision about CI minutes, not just disk. **Deleting from the host filesystem is not a shortcut.** Removing directories under the data root by hand, rather than through the daemon, leaves the daemon's metadata pointing at layers that no longer exist and can corrupt the installation. Every reclaim goes through the API. ## Step 4 — leave the host better than you found it The incident answer ends with a prevention answer: configure log rotation on the log driver so the invisible bucket stops growing, put an age-filtered prune on a timer instead of waiting for the next page, and monitor free space with an alert threshold that fires while there is still room to act.