In PostgreSQL, what do pg_basebackup's -X none, -X fetch and -X stream do, and why is a copy without its WAL unusable?
answer
- files copied over hours, not one instant
- replay from start checkpoint to backup end
- second connection plus temporary slot
- fetch needs the source to keep WAL
basics
~20 sCopied files are consistent only after replaying the WAL written from the backup's start to its end. -X stream (default) receives that WAL live, -X fetch collects it at the end, and -X none leaves it to the WAL archive.
solid answer
~40 sThe copied files come from different moments, so a server can only make them consistent by replaying the WAL written between the backup's start checkpoint and its end. `-X` decides where that WAL comes from. `stream`, the default, opens a second replication connection, receives WAL while files copy, and uses a temporary replication slot so the server keeps it; it needs two WAL senders. `fetch` collects the WAL at the end over the same connection, so the source must still have it: `wal_keep_size` (default 0) has to cover the whole backup, or the backup fails. `none` includes no WAL; the restore must fetch it from the archive with `restore_command`, and `pg_basebackup` waits at the end until that WAL is archived. With `stream` or `fetch` the backup is self-contained.
go deeper
Recall the three -X methods by name and that stream is the default; a backup without its WAL cannot start.
Explain why copied files are inconsistent until WAL from the start checkpoint to the backup end is replayed, and what each method needs from the server.
Pick a method per situation: fetch's dependence on wal_keep_size, stream's temporary slot and two WAL senders, none when the archive is trusted, and --target's restriction.
Decide where the WAL contract lives for the whole estate: self-contained nightly copies versus copies that are only valid together with the archive.
## Why a base backup needs WAL `pg_basebackup` copies files while the cluster keeps writing. A copy of the 2 TB insurance-claims cluster takes hours, so the first file and the last file are copied at very different moments, and a page can even be copied while it is being written. The files alone are **not consistent**. What makes them consistent is **WAL replay**: starting from the checkpoint at which the backup began, the server replays every WAL record written until the backup ended. During backup mode PostgreSQL forces `full_page_writes` on, so the WAL holds full page images that let replay repair torn pages. A base backup is therefore the files **plus** the WAL range from backup start to backup end. The `-X` option (`--wal-method`) decides where that WAL comes from. ## The three methods | Method | How the WAL arrives | What can go wrong | |---|---|---| | `none` (`n`) | not included | without a working archive, the copy can never start | | `fetch` (`f`) | at the end, over the same connection | the server may have recycled segments first; the backup then fails and is unusable | | `stream` (`s`, default) | live, over a second replication connection | needs two WAL senders; the client must keep up | - **`none`** leaves recovery to the WAL archive: the restored server reads segments through `restore_command`. Because the copy depends on the archive, `pg_basebackup` asks the server to wait at the end until the backup's WAL has been archived. Choose it only when archiving is set up and monitored. - **`fetch`** collects the WAL after the files. The source must still hold it, which means setting `wal_keep_size` (default 0, in megabytes) high enough to cover the backup's whole duration. On a busy 2 TB cluster that is a guess, and a wrong guess loses the backup at its very last step. In tar format the WAL goes inside `base.tar`. - **`stream`** opens a second connection and receives WAL while the files are copied. Unless told otherwise it creates a **temporary replication slot**, so the server keeps WAL until the client has it. `-S name` uses a named slot (`-C` creates it), and `--no-slot` skips the slot when none are free. In tar format the WAL lands in `pg_wal.tar`. With `fetch` or `stream` the backup is **self-contained**: a server can start on it without any archive. Recovering to a moment after the backup ended still needs archived WAL, which is the point-in-time recovery subject. ## Interactions worth knowing 1. `--target=server:/path` or `--target=blackhole` makes the server write the backup, but streaming is done by the `pg_basebackup` client. So `--target` requires `-X fetch` or `-X none`. 2. Server-side compression never compresses the streamed WAL. Client-side gzip compresses `pg_wal.tar`; other client-side algorithms leave it uncompressed. 3. `--max-rate` always throttles the data files but throttles WAL only with `fetch`. 4. A standby cannot switch WAL segments at the end of a backup. With `-X none` and little write activity on the primary, the run can wait a long time for the last segment unless someone runs `pg_switch_wal()` on the primary. ## -R and what comes after `-R` (`--write-recovery-conf`) creates `standby.signal` and appends `primary_conninfo`, plus `primary_slot_name` when `-S` was used, to `postgresql.auto.conf` (inside `base.tar` in tar format). That prepares the copy to start as a streaming standby. Operating that standby is a replication topic, not a backup one. ## For the claims cluster ```bash pg_basebackup -h claims-db1 -U claims_backup \ -D /backup/claims/2026-10-06 -F tar -z \ -X stream -P ``` `-X stream` matches the default, and writing it out documents the backup's WAL contract in the job definition. The nightly copy can start on its own; the archive still matters for anything beyond its end.
- What does pg_basebackup -R add to the copy, and what does it not do?It creates `standby.signal` and appends `primary_conninfo`, plus `primary_slot_name` if `-S` was given, to `postgresql.auto.conf` in the backup. It does not start anything or create a permanent slot by itself; it only prepares the copy so that, once started, it runs as a streaming standby of the source.
- With -X stream, why would you pass -S with a named slot instead of relying on the default temporary slot?The temporary slot disappears when the backup ends. If the copy will become a standby, a named slot (created with `-C`) keeps WAL from the end of the backup until that standby connects with the same `primary_slot_name`. The cost: an abandoned permanent slot keeps WAL on the primary indefinitely, so drop it if the standby never arrives.
- Is the streamed WAL compressed when you ask pg_basebackup for compression?Not with server-side compression; the streamed WAL is never compressed there. With client-side gzip, `pg_wal.tar` is gzip-compressed; with client-side lz4 or zstd it stays uncompressed. Use `-X fetch` or client-side gzip if WAL size in the backup matters.
- -X fetch is safe because the server keeps all WAL written during a backup
- a -X none backup can be started on its own without any WAL archive
- -X stream sends WAL over the same connection as the data files
- raising max_wal_size guarantees the WAL for -X fetch is still there
- --target=server works together with the default WAL method