With pgBackRest, how does restore --delta shorten rebuilding a PostgreSQL data directory that already holds most of its files, and when should you use it?
answer
- empty directory by default
- hash each file, keep matches
- removes what the manifest lacks
- server must be stopped
- force plus delta trusts metadata
basics
~20 spgBackRest restore --delta keeps existing files whose SHA-1 matches the backup manifest, restores only mismatched or missing files, and deletes files the backup lacks. Rebuilding a mostly intact data directory then copies a fraction of the data.
solid answer
~40 sA plain pgBackRest `restore` expects an empty data directory and fails with 'unable to restore to path ... because it contains files', hinting at `--delta`. With `--delta`, pgBackRest computes a SHA-1 hash of every existing file and compares it with the backup manifest. It restores only files that differ or are missing and removes files the manifest does not list, which disposes of divergent changes. PostgreSQL must be stopped, since a `postmaster.pid` aborts the restore, and you can therefore set `process-max` high. Adding `--force` makes the comparison size and timestamp, which is faster but trusts metadata. Block incremental makes it faster still, because single blocks can be fetched. Use it to rebuild a former primary or a broken standby (`--type=standby`), to refresh a test copy, or after damage to a few files.
code
bash · 7 lines# former primary, now to follow the new primary
sudo systemctl stop postgresql@18-main
sudo -u postgres pgbackrest --stanza=analytics --repo=1 \
--delta --type=standby --process-max=16 \
--recovery-option=primary_conninfo='host=analytics-db-2 user=replicator' \
restore
sudo systemctl start postgresql@18-maingo deeper
Recall that restore needs an empty directory unless --delta is used, and that PostgreSQL must be stopped first.
Explain the mechanism: SHA-1 per file against the manifest, restore the mismatches, delete what the backup lacks, and how --force changes the test.
Show you can cut real restore time: delta plus high process-max, block incremental, the nearest repository, and async archive-get for replay.
Treat restore time as an engineered number: repository placement, backup recency and delta reuse determine it, and each needs rehearsal.
## The default: an empty data directory A plain `pgbackrest restore` assumes the target data directory (and any tablespace directories) are **empty**. If files are present it stops with `unable to restore to path '...' because it contains files` and suggests `--delta`. Two guards come first in every restore: - If `postmaster.pid` exists, pgBackRest refuses with 'unable to restore while PostgreSQL is running'. **The server must be stopped.** - With `--delta` or `--force`, if the path contains neither `PG_VERSION` nor a `backup.manifest`, pgBackRest disables both options, so it cannot wipe a directory that is not a PostgreSQL data directory. For a 5 TB analytics cluster, emptying the directory and copying 5 TB back from an object store takes hours. That is time a delta restore can often avoid. ## What --delta does 1. It reads the chosen backup's manifest. By default that is the latest backup in the first repository that holds backups. 2. It computes a **SHA-1 hash** of each file already in the data directory. 3. It keeps files whose hash matches the manifest and restores files that differ or are missing. 4. It **removes files the manifest does not list**, discarding changes made after the backup, such as a former primary's divergent writes. 5. It writes `restore_command` (calling `archive-get`) into `postgresql.auto.conf`, plus `recovery.signal`, or `standby.signal` with `--type=standby`. It does **not** start PostgreSQL, so you can review the files first. ## Delta, force, or both | restore options | existing files | how a file is judged | trade-off | |---|---|---|---| | none | must be absent | not applicable | fails if the directory has files | | `--delta` | compared, kept if equal | SHA-1 checksum | safe; reads every local file once | | `--delta --force` | compared, kept if equal | size and timestamp | faster; trusts file metadata | | `--force` | overwritten | no comparison | rewrites everything | ## Making the restore fast - **`process-max`**: PostgreSQL is down during the restore, so you can use far more processes than you would allow during a backup. Hashing and copying both parallelise. - **Block incremental** (`repo-block`): a delta restore can fetch only the changed blocks of a file, not the whole file. - **Repository choice**: pass `--repo` to restore from the closest or cheapest repository, not just the first one configured. - **WAL fetching**: with `archive-async=y`, `archive-get` keeps a local queue (`archive-get-queue-max`, default 128MiB) filled ahead of recovery. That matters when WAL comes from a high-latency object store. Recovery time also includes WAL replay after the copy. A delta restore shortens the copy, but it does not shorten replay. Picking a more recent backup does. ## When to reach for it - **Rebuilding a former primary as a standby** after a failover, when most of its files still match a recent backup. `pg_rewind` is core's alternative and belongs to the promotion topic. - **Refreshing a standby** that fell too far behind or lost WAL it needed. - **Repeated restores** into the same staging or test directory. - **Localised damage**, where a few files were corrupted or deleted. ## Worked example: the old primary after a failover The analytics cluster failed over from `analytics-db-1` to `analytics-db-2`. The old primary still holds almost all of its 5 TB, plus a few minutes of writes the new primary never received. 1. Make sure the old primary's PostgreSQL is stopped and fenced, so it cannot accept writes. 2. Run `restore --delta --type=standby` against the nearby repository, with a high `process-max` and a `--recovery-option` that sets `primary_conninfo` to the new primary. 3. pgBackRest hashes the 5 TB locally, fetches only the files or blocks that differ from the backup, and deletes files the backup does not know. 4. Start PostgreSQL. It replays archived WAL through `archive-get` up to the new timeline, then streams from the new primary. The network transfer is the changed fraction, not 5 TB, which is the whole point. ## Limits - Every local file is still read once to hash it, so the saving is in network transfer and writes, not local reads. - `--delta --force` can miss a changed file whose size and timestamp happen to match. Prefer plain `--delta` unless time is critical. - The removal step is a feature, but it is destructive: anything in the directory that the backup lacks is gone.
- Why does a pgBackRest delta restore of a former primary remove files, and why is that desirable?Delta deletes any file that is not in the backup manifest, and it restores files whose checksum differs. After a failover, the old primary may hold writes made after the new primary's timeline branched. Removing and replacing those files brings it back to the backup's state, and WAL replay then carries it forward on the new timeline.
- With pgBackRest, when would you add --force to a --delta restore, and what do you give up?When local hashing time dominates, for example on a very large directory with slow disks, and you trust that file sizes and timestamps reflect real changes. pgBackRest then compares size and timestamp instead of SHA-1. You give up certainty, because a file changed without a size or timestamp change would be kept.
- pgBackRest restore can run while PostgreSQL is up as long as you pass --delta.
- --delta only adds missing files and leaves extra files untouched.
- Delta restore compares file sizes and timestamps by default.
- --force on its own keeps any files that already match the backup.
- pgBackRest restore starts PostgreSQL automatically once files are copied.
- A delta restore also shortens the WAL replay that follows it.