A systemd timer you deployed is not firing when you expected. How do you check what systemd thinks your OnCalendar= expression means, when the timer last and next elapses, and whether the problem is in the timer or in the service it triggers?

## Split the problem in two before you touch anything A timer-driven job has two halves that fail for different reasons: the timer may not be arming or not elapsing when you think, or the timer may be firing perfectly and the service may be failing. The first question to answer is which of those you are in, because it halves the search space immediately. `systemctl list-timers` answers it in one line: if the `LAST` column shows a recent elapse, the timer is doing its job and you should stop looking at the schedule. ## Step 1 — does the expression mean what you meant? ```bash systemd-analyze calendar 'Mon *-*-* 02:00:00' systemd-analyze calendar '*-*-* 0/6:00:00' systemd-analyze calendar daily ``` The output gives the original form, systemd's normalized form, and the next elapse — printed both in local time and in UTC. Two classes of bug fall out here. Syntax you got wrong (a mis-ordered field, a repetition you thought meant something else) shows as a parse error or an obviously wrong normalization. And the local/UTC pair exposes the time-zone assumption: calendar expressions are local time unless suffixed with `UTC`, so a host whose zone is not what you assumed will fire at a different absolute moment. `timedatectl` confirms the host's zone and whether the clock is synchronized. `systemd-analyze timespan 15min` does the equivalent sanity check for the monotonic directives and for `RandomizedDelaySec=`. ## Step 2 — is the timer actually armed? ```bash systemctl daemon-reload systemctl list-timers --all systemctl status backup.timer ``` `list-timers` without `--all` shows only active timers; `--all` includes inactive ones, which is how you discover that the timer exists but was never started. The columns are `NEXT`, `LEFT`, `LAST`, `PASSED`, `UNIT` and `ACTIVATES`. Read them in that order: a plausible `NEXT` proves arming and schedule; `LAST`/`PASSED` prove whether it has ever run; `ACTIVATES` proves the pairing resolved to the service you intended rather than to a name that does not exist. `systemctl status backup.timer` adds the state — `active (waiting)` is healthy — and a `Trigger:` line with the next firing. It is also where a bad directive surfaces: an unparseable `[Timer]` setting makes the unit fail to load and status reports the offending line. `systemd-analyze verify /etc/systemd/system/backup.timer` performs that check without deploying. The recurring causes at this step, in rough order of frequency: nobody ran `daemon-reload` after writing the files; the `.service` was enabled instead of the `.timer`; the timer was enabled but never started and the host has not rebooted; the unit is masked; the timer's name does not pair with the service's and no `Unit=` was set. ## Step 3 — the timer fires and the job still does not happen If `LAST` shows recent elapses, hand the investigation to the service half: ```bash systemctl status backup.service journalctl -u backup.service --since today ``` Status reports the outcome of the most recent run — a `oneshot` unit that succeeded shows `inactive (dead)` with a zero exit, while a failure shows `failed` with the exit status or signal. The journal carries whatever the job wrote to stdout and stderr. The highest-value move here is to run the job yourself: ```bash systemctl start backup.service ``` This is not the same as running the script in your shell. It executes the unit with the same `User=`, `WorkingDirectory=`, environment, resource limits and cgroup that the scheduled run gets, which is precisely where the "works when I run it, fails on schedule" class of bug lives — a `PATH` the script assumed, a credential only present in your login session, a relative path resolved from your home directory. ## Step 4 — the subtler cases - **The job runs but takes longer than the interval.** systemd will not start a second copy while the unit is still active, so triggers appear to be skipped. `LAST` advances less often than the schedule suggests, and status shows a long-running activation. - **A randomized delay is in play.** With `RandomizedDelaySec=` set, the elapse legitimately drifts from the nominal time; `systemctl show backup.timer -p RandomizedDelayUSec` confirms what is configured. - **DST or a clock step.** A calendar time inside a daylight-saving transition can be skipped or duplicated. `journalctl` timestamps around the transition make it obvious. - **A persistent timer already caught up.** With `Persistent=true`, the run you were waiting for may have happened right after boot rather than at the nominal hour; the stamp under `/var/lib/systemd/timers` and the `LAST` column agree on when. ## The order matters Expression, then arming, then the service. Each step is one command, each rules out a whole class of cause, and running them in that order stops you from tuning a schedule that was firing correctly all along.