On a systemd host, /var is nearly full and /var/log/journal accounts for several gigabytes. How do you reclaim the space now with journalctl, and how do you cap the journal so it cannot grow that far again?

## Measure before you delete ```bash journalctl --disk-usage # Archived and active journals take up 3.9G in the file system. du -sh /var/log/journal/* ``` `--disk-usage` counts both archived and active files and names the path, which incidentally tells you whether you are looking at the persistent journal or the runtime one. If it reports megabytes while `du` on `/var` shows gigabytes, the journal is not your problem and you should go look at `/var/lib` instead — a mistake worth ruling out in the first thirty seconds of a full-disk page. ## How journald manages its own files journald is its own rotation manager; `logrotate` has nothing to do with it. It writes to an **active** journal file per journal set, and when that file hits `SystemMaxFileSize=` — or when rotation is requested — it renames it to an archived file (`system@<seq>.journal`) and starts a new active one. Retention is then enforced by deleting whole archived files, oldest first, until the configured limits are satisfied. This is why the limits are approximate: the granularity of deletion is a file, not an entry. ## Reclaiming space now Three vacuum forms, all of which operate on **archived files only**: ```bash journalctl --vacuum-size=500M # keep the journal set under this total journalctl --vacuum-time=7d # drop entries older than this journalctl --vacuum-files=5 # keep at most this many files per set ``` The trap that costs people an outage-minute: because the active file is exempt, a journal that is one enormous active file will barely shrink. Rotate first: ```bash sudo journalctl --rotate sudo journalctl --vacuum-time=2d journalctl --disk-usage ``` `--rotate` closes the active file and archives it, so the subsequent vacuum can consider it. The two can also be given in a single invocation. Note also that vacuum limits are applied per journal set — the system journal and each user journal are separate sets — so `--vacuum-size=500M` is not a global 500 MB budget on a host with busy user journals. Resist deleting files under `/var/log/journal` by hand. It usually works, but the vacuum path lets journald keep its own bookkeeping consistent and refuses to break the active file; `rm` does neither, and a truncated or half-deleted set is a worse morning. ## Capping it permanently The settings live in `/etc/systemd/journald.conf` or, better, a drop-in: ```ini # /etc/systemd/journald.conf.d/limits.conf [Journal] SystemMaxUse=1G SystemKeepFree=2G SystemMaxFileSize=128M SystemMaxFiles=20 ``` - `SystemMaxUse=` — the total the persistent journal may occupy. Unset, it defaults to **10% of the filesystem's size, capped at 4 GB**, which on a 200 GB `/var` means journald was entitled to every one of those gigabytes it took. - `SystemKeepFree=` — the free space journald will leave on that filesystem, defaulting to **15% of it**. This is the one that matters when `/var` is shared with something else that needs room, because journald will delete its own history to honour it. - `SystemMaxFileSize=` — the size at which the active file rotates. Keep it well under `SystemMaxUse=`; a single file that is most of the budget makes retention lumpy, since deleting one file drops a huge slice of history at once. - `SystemMaxFiles=` — a cap on file count per set. Each has a `Runtime*` twin (`RuntimeMaxUse=`, `RuntimeKeepFree=`, …) governing the volatile journal in `/run` — worth setting too, because that one consumes RAM. journald applies whichever limit binds first. Reload with `systemctl restart systemd-journald`, and confirm the effective configuration with `systemd-analyze cat-config systemd/journald.conf`, since a vendor drop-in may already be setting what you think you are setting. ## Then ask why it filled Capping the journal makes the disk safe but throws away the question. A journal that grew to gigabytes in days is usually one unit in a loop — a crash-restart cycle, a stack trace per request, a debug level left on after an investigation. Find it before you walk away: ```bash journalctl --since "1 day ago" -o json --output-fields=_SYSTEMD_UNIT | \ sort | uniq -c | sort -rn | head ``` Or, more cheaply, look at what `systemctl list-units --failed` and the restart counters say. The fix is often in the unit — a `Restart=` policy that hides a crash loop, or a log level — and not in journald at all. Note the second-order effect too: capping `SystemMaxUse=` on a chatty host silently shortens your retention window, so the incident three days ago may no longer be in the journal when you go looking. ## Version note These directive names are stable across current systemd releases; `--vacuum-files=` is the newest of the three vacuum forms and is present on every systemd shipping with a currently supported distribution.