skip to content

After a nightly pg_basebackup of a PostgreSQL 18 cluster, what does pg_verifybackup check against backup_manifest, and what can it still miss?

level: seniorimportance: should knowfreq 38%

answer

  1. a JSON list written by the server
  2. four stages, last one reads WAL
  3. tar needs one stage turned off
  4. bytes match is not a restore

basics

~20 s

pg_verifybackup checks the manifest itself, then missing, extra or resized files, then each file's checksum, then parses the needed WAL with pg_waldump. It cannot prove the backup restores: wrong-but-valid WAL and anything outside the manifest pass.

solid answer

~40 s

`pg_basebackup` stores a `backup_manifest`, a JSON list of every non-WAL file with size and checksum (CRC32C by default), the system identifier and the WAL range needed for recovery. `pg_verifybackup` checks it in four stages: the manifest is readable, self-consistent and matches `pg_control`; no file is missing, extra or resized (a few expected files are ignored); every checksum matches; and the needed WAL parses through `pg_waldump`. PostgreSQL 18 also verifies tar-format backups, but WAL parsing must then be skipped with `-n`. It cannot prove the backup works: it never replays WAL, so well-formed but wrong WAL passes, and CRC32C does not stop deliberate tampering. Corruption already on the primary is caught differently, by page checksums verified while the backup is sent.

go deeper

for a junior

Recall that pg_basebackup writes a backup_manifest by default and that pg_verifybackup checks a backup against it.

for a middle

Walk through the four stages: manifest sanity, file list, file checksums and WAL parsing, plus the tar-format rule with -n in 18.

for a senior

Separate page checksums from manifest checksums, choose SHA versus CRC32C with storage of the manifest, and state plainly what a clean run cannot prove.

for a principal

Place manifest verification in the cost ladder of backup assurance and decide how often expensive full restores must back it up.

## The backup manifest Unless you pass `--no-manifest`, the server sends a **backup manifest** with every `pg_basebackup` run, stored as `backup_manifest` in the backup. It is a JSON document recording: - the cluster's **system identifier**; - every file in the backup except WAL files, with size, modification time and a checksum; - the **WAL ranges** (timeline, start and end LSN) needed to recover the backup; - a SHA-256 checksum of the manifest itself. `--manifest-checksums` picks the per-file algorithm: `NONE`, `CRC32C` (the default), `SHA224`, `SHA256`, `SHA384` or `SHA512`. CRC-32C is cheap and catches accidental damage. A SHA algorithm costs noticeably more CPU on a 2 TB copy, but only a cryptographic digest helps against deliberate tampering, and only if the manifest is stored where an attacker cannot rewrite it too. The manifest is also what `pg_basebackup --incremental` takes as input, so a backup taken with `--no-manifest` can be neither verified nor used as an incremental base. ## What pg_verifybackup checks 1. **Read the manifest.** A missing, unreadable or malformed manifest, one failing its own checksum, or one whose system identifier differs from the backup's `pg_control`, is a fatal error. 2. **Compare the file list.** Extra files, missing files and size differences are reported. It ignores `postgresql.auto.conf`, `standby.signal`, `recovery.signal`, `backup_manifest` and anything in `pg_wal`, because they are expected to change or are not listed. 3. **Checksum every file** against the manifest, skipping files that already failed stage 2. `-s` skips this stage, leaving only names and sizes. 4. **Parse the needed WAL** by running `pg_waldump` over the range the manifest names. `-n` (`--no-parse-wal`) skips it; `-w` points at WAL stored outside `pg_wal`. Other options: `-m` reads a manifest kept elsewhere, `-e` stops at the first problem, `-i` ignores a path you added on purpose. ## Tar format and versions | Backup | PostgreSQL 17 | PostgreSQL 18 | |---|---|---| | Plain format | verified | verified | | Tar format, compressed or not | not supported | verified, with WAL parsing turned off by `-n` | Without `-n`, version 18 stops on a tar backup with "pg_waldump cannot read tar files". WAL parsing is version-specific, so use the `pg_verifybackup` of the backup's major version; the file checks work with manifests from any version. ## Two different checksums - **Data page checksums** belong to the cluster. When the cluster has them (`initdb` enables them by default since 18), pages are checked while the base backup is sent. A failure makes `pg_basebackup` exit non-zero but keeps the backup, and is counted in `pg_stat_database.checksum_failures`. `--no-verify-checksums` turns this off. It catches corruption **already present on the primary**. - **Manifest file checksums** are checked by `pg_verifybackup`. They catch damage that happened **after** the copy: storage, transfer, an edit. ## What a clean run does not prove A clean run means the bytes match what the server sent and the needed WAL parses. It does not replay that WAL, so: - a server bug that wrote well-formed but wrong WAL records passes; - a backup rewritten together with a manifest stored beside it passes, since the checksums can simply be recomputed; - extra WAL beyond the needed range is not checked at all; - `pg_combinebackup` output must be verified separately, from its own new manifest. Only a test restore proves a backup works. For the claims cluster the nightly job verifies right after the copy, and the manifest is copied to separate storage: ```bash pg_basebackup -h claims-db1 -U claims_backup -X stream \ -D /backup/claims/2026-10-06 --manifest-checksums=SHA256 cp /backup/claims/2026-10-06/backup_manifest /vault/claims/manifest-2026-10-06 pg_verifybackup -m /vault/claims/manifest-2026-10-06 /backup/claims/2026-10-06 ```

  • Why move a PostgreSQL backup_manifest to separate storage before running pg_verifybackup -m on it?
    Someone who can alter the backup can also rewrite a manifest stored next to it and recompute its checksums. Keeping the manifest elsewhere, ideally with SHA-256 file checksums, means tampering with the backup no longer matches the trusted manifest.
  • pg_basebackup exited non-zero with page checksum failures. Is the PostgreSQL backup gone, and what does the failure tell you?
    The backup is kept, as if `--no-clean` were given. The failures mean data pages on the source cluster were already corrupt when read, so the primary needs investigation; `pg_stat_database.checksum_failures` shows which database. Verifying the backup with `pg_verifybackup` would not catch this, because the manifest checksums the corrupt bytes as sent.

  • pg_verifybackup restores the backup and replays WAL to prove it works
  • a clean pg_verifybackup run means a test restore is unnecessary
  • CRC32C manifest checksums protect against deliberate tampering
  • PostgreSQL 17's pg_verifybackup can check tar-format backups
  • manifest checksums detect pages that were already corrupt on the primary

Assumes PostgreSQL 18; pg_verifybackup verifies tar-format backups only since 18, and initdb enables data checksums by default since 18.