With pgBackRest, how do full, diff and incr backups depend on each other, and how do repo1-retention-full and repo1-retention-diff decide what expire removes?
answer
- default type is incr
- diff from last full only
- size and timestamp, not blocks
- expire runs after each backup
- N+1 fulls before expiry
basics
~20 sA pgBackRest full stands alone, a diff holds files changed since the last full, and an incr holds files changed since the last backup of any type. Expire removes the oldest fulls beyond repo1-retention-full, along with their diffs and incrs.
solid answer
~50 sIn pgBackRest a `full` copies every file, a `diff` copies files changed since the last **full**, and an `incr` copies files changed since the last backup of **any** type. The default `--type` is `incr`. Restoring a diff needs its full, and restoring an incr needs every backup back to the prior diff or full. Change is judged per file by size and timestamp, unless delta checksums are on. Retention is per repository, and `expire` runs after every successful backup (`expire-auto` defaults to on). `repo1-retention-full` keeps N fulls by count, or by days with `repo1-retention-full-type=time`, but only after a new full completes, so N+1 exist briefly. Expiring a full removes its diffs and incrs. `repo1-retention-diff` counts fulls as diffs, and incrs have no retention of their own. Archived WAL older than the oldest retained backup is expired with it. If `repo1-retention-full` is unset, pgBackRest warns and expires nothing.
code
ini · 8 lines[global]
repo1-path=/var/lib/pgbackrest
repo1-retention-full-type=count
repo1-retention-full=2
repo1-retention-diff=6
[analytics]
pg1-path=/var/lib/postgresql/18/maingo deeper
Recall the three types: full stands alone, diff builds on the last full, incr builds on the last backup of any kind, and incr is the default type.
Explain the restore chain each type creates, how expire removes dependents with their full, and why a count of two briefly means three fulls.
Show you can size a repository and a PITR window from a schedule, and know why tightening WAL retention silently removes point-in-time recovery.
Weigh restore time against storage: more fulls or diffs shorten restore chains but cost space, and retention differs per repository by purpose.
## Three backup types and what each depends on pgBackRest offers three values for the `backup` command's `--type`, and the **default is `incr`**: | type | copies files changed since | label ends in | a restore also needs | |---|---|---|---| | `full` | nothing; copies every file | `F` | nothing else | | `diff` | the last **full** backup | `D` | that full | | `incr` | the last backup of **any** type | `I` | every backup back to the prior diff or full, and that full | If you ask for a diff or an incr when no full exists, pgBackRest takes a full instead. A diff is restored by copying its own files plus the unchanged files from its full. An incr can depend on a chain of earlier incrs, a diff and a full, and every link in that chain must be valid. A later backup records references to files it did not copy, so `info` shows both a **backup size** (this backup's own files) and a **backup set size** (everything needed to restore it). ## How pgBackRest decides a file changed Without `--delta`, a diff or incr compares each file's **size and modification timestamp** with the prior backup's manifest. Some anomalies make that comparison untrustworthy, such as a timestamp earlier than the prior backup's, or a changed size with an unchanged timestamp. When pgBackRest finds one, it logs a warning and switches that backup to **delta checksum** mode, comparing checksums instead. The unit of change is the **whole file**. PostgreSQL stores relations in segment files of up to 1 GB, so a single changed row recopies its whole segment unless block incremental (`repo-block`) is enabled. ## What expire removes Retention is configured **per repository** (`repo1-...`, `repo2-...`) and applied by `expire`. Because `expire-auto` defaults to on, expire runs automatically after every successful backup. 1. **`repo1-retention-full`** with `repo1-retention-full-type=count` (the default) keeps that many fulls. Expiry happens only after a new full **completes**. With a value of 2, the third full must finish before the oldest goes, so size the repository for N+1. 2. With `repo1-retention-full-type=time` the value is a number of days. A full is removed only if a newer full at least that old would remain. With 30 days and fulls aged 25 and 35 days, both stay. 3. Expiring a full also removes **every diff and incr that depends on it**. 4. **`repo1-retention-diff`** keeps that many differentials, and fulls count as differentials for this purpose. Expiring a diff removes the incrs that depend on it. 5. Incrementals have **no retention option of their own**. They always leave with their full or their diff. 6. If `repo1-retention-full` is not set, pgBackRest warns that the repository may run out of space and expires no fulls. ## WAL archive retention By default pgBackRest keeps archived WAL from the oldest retained backup onward and deletes older WAL, because only WAL written after a backup can roll that backup forward. `repo1-retention-archive-type` (default `full`) and `repo1-retention-archive` can expire WAL more aggressively. Each older backup then keeps only the WAL it needs to reach consistency, and point-in-time recovery between those backups is lost. The reference recommends leaving these options at their defaults. Running `expire --dry-run` shows what would be removed without deleting anything. ## A worked schedule Take the analytics cluster with a full every Sunday, a diff every other day, `repo1-retention-full=2` and `repo1-retention-diff=6`: - Restoring any day needs at most two backups, the diff and its full, which keeps restore time predictable. - When the third Sunday full completes, the oldest full, its diffs and the WAL before the next full are expired. - Right after that full, the point-in-time window reaches back about one week, to the older retained full. It grows to about two weeks just before the next full. It is not a flat two weeks. ## Mistakes interviewers listen for - Saying a diff is relative to the previous diff. That describes an incr. - Expecting incrementals to be kept or expired on their own schedule. - Forgetting the extra full that exists briefly before expiry. - Shrinking WAL retention to save space without noticing it removes point-in-time recovery. - Leaving `repo1-retention-full` unset and finding a full repository months later.
- With pgBackRest, why might a diff backup log 'enabling delta checksum' and run longer than usual?Comparing manifests turned up a timestamp anomaly: a file with an older timestamp than in the prior backup, or a file whose size changed while its timestamp did not. This happens after clock changes or when files come back with old timestamps. pgBackRest then compares checksums for that whole backup instead of trusting size and timestamp. It reads every file, which is slower but cannot miss a change.
- With weekly fulls and repo1-retention-full=2 in pgBackRest, how far back can you recover to a point in time?About one to two weeks, not a flat two. When the third full completes, the oldest full and the WAL before the next full are expired, so the window starts at the older retained full, roughly a week back. It then grows by a day each day until the next full completes and expiry trims it again.
- How does repo1-retention-full-type=time with repo1-retention-full=30 decide whether to expire a full?The value means days. A full is expired only if at least one other full that is 30 days old or older would remain. With fulls aged 25 and 35 days, neither is expired, because removing the 35-day full would leave no full that covers 30 days. Archived WAL older than the oldest remaining full is expired too.
- A pgBackRest diff contains the changes since the previous differential.
- Incremental backups have their own retention setting and expire on their own.
- With repo1-retention-full=2 the repository never holds more than two fulls.
- Expire must be scheduled separately or pgBackRest never removes anything.
- Lowering repo1-retention-archive saves space at no cost to recovery.
- If repo1-retention-full is unset, pgBackRest keeps two fulls by default.