skip to content

Hybrid Persistence

You will learn the default modern setup: an RDB preamble inside the AOF, giving snapshot-fast reloads with append-log durability. Interviewers ask 'RDB or AOF?' expecting you to answer 'both, and here is how the hybrid file works'.

part ofRedisoverview, primer and where to startread it →
on this pageshow

questions

4

Redis has a configuration directive called `aof-use-rdb-preamble`. What does enabling it change about the append-only file Redis writes and reads, and what do you gain from it?

level: middleimportance: must knowfreq 45%

answer

  1. RDB body + AOF tail in one file
  2. default yes since Redis 4.0
  3. load = RDB loader then command replay
  4. speed feature, not a durability feature
  5. 7.0: base.rdb + incr.aof + manifest

basics

~20 s

With it on, a rewritten append-only file starts with the whole dataset in compact binary snapshot format, then appends later write commands as normal text. One file, snapshot body plus command tail. You gain much faster loading and a smaller file; durability is unchanged.

solid answer

~50 s

The append-only file (AOF) is normally a log of every write command in RESP text form; replaying it rebuilds the dataset. With `aof-use-rdb-preamble yes` (the default since Redis 4.0), when Redis rewrites the AOF the child does not re-emit commands — it serializes the current dataset in the binary RDB snapshot format and writes that as the **head** of the new AOF. Every write accepted after that point is appended behind it as ordinary RESP commands, so the file is "RDB body + AOF tail". On startup Redis sniffs the first five bytes: the `REDIS` magic means load the head with the RDB loader, then hand the remainder to the AOF parser. The win is load time and file size — parsing a packed binary image beats re-executing millions of commands. What it does **not** change is durability: the tail is still governed by `appendfsync` (default `everysec`), so the exposure window is the same as before.

code

text · 4 lines
text
appendonly yes
appendfsync everysec
aof-use-rdb-preamble yes    # default since 4.0
appenddirname "appendonlydir"   # Redis 7.0+

go deeper

for a junior

Know that the AOF can begin with a compact binary copy of the data and continue with normal commands, and that this makes restarts faster.

for a middle

Explain the rewrite producing an RDB body, the RESP tail after it, the REDIS magic driving the two-phase load, and that appendfsync still owns durability.

for a senior

Add operational consequences: faster failover and restart, loss of human-readable forensics for the body, RDB version coupling on downgrade, and how the 7.0 base/incr layout expresses the same idea.

for a principal

Frame it as decoupling recovery cost from durability granularity, and reason about where that changes fleet-level choices — restart budgets, failover time objectives, and whether AOF becomes viable on datasets where command replay was previously too slow.

## The two file formats Redis can write Redis persists in two shapes. An **RDB file** is a compact binary point-in-time image of the dataset: keys, values in their internal encodings, expirations, optionally LZF-compressed, with a CRC64 checksum. An **AOF (append-only file)** is a log: every write command the server accepted, written out in the same RESP protocol text the client sent, so recovery means re-executing the log against an empty database. The log form is attractive for durability (you can fsync it every second, or every write) but bad for recovery cost. A list built by a million `RPUSH` calls is a million commands to parse and execute on load; the same list in RDB form is one key, serialized once. ## What the preamble actually does `aof-use-rdb-preamble yes` — the default since **Redis 4.0** — changes what the AOF *rewrite* child produces. Instead of walking the dataset and emitting the minimal set of commands that would rebuild it, the child serializes the dataset **in RDB format** and writes it as the first section of the new AOF. From the instant the rewrite starts, newly accepted writes are appended after that section as ordinary RESP commands. So a live hybrid AOF looks like: ``` [ REDIS0011 ...binary snapshot of the dataset at rewrite time... ] [ *3\r\n$3\r\nSET\r\n... ] <- every write since the rewrite ``` This is one file with two formats inside it, not two files. On load, Redis reads the first bytes; the `REDIS` magic tells it to run the RDB loader over the head, and when that section ends it switches to the AOF parser and replays the tail commands. If the preamble is disabled the file simply starts with a RESP command (typically `SELECT`) and Redis parses the whole thing as commands. ## What you gain - **Restart and failover speed.** Loading is dominated by the binary section, which is bulk deserialization rather than command dispatch. On multi-gigabyte datasets this is commonly several times faster than a pure command replay. - **Smaller files.** Encodings are preserved and the body can be compressed; a command log for the same data is far more verbose. - **Cheaper rewrites in aggregate**, because the produced base is smaller and therefore faster to write and read. ## What it does not change - **Durability.** The head is written once at rewrite time; everything after it is the ordinary command tail, still flushed according to `appendfsync` (`everysec` by default, `always` per write, `no` to leave it to the OS). Hybrid persistence is a *load-speed* feature that lets you keep second-level durability without paying command-replay recovery cost. Saying "the preamble makes the AOF durable" is wrong. - **Snapshot backups.** The RDB *body inside the AOF* is not your `dump.rdb`. Configured save points still produce a separate snapshot file, and that file is still what you copy off the box for backups. ## Costs and caveats - The head is opaque binary. You can no longer read or grep the beginning of an AOF, and you cannot hand-edit out a poisonous command that predates the last rewrite. `redis-check-aof` understands the hybrid layout and can truncate a torn *tail*, but the body is not text you can repair by eye. - The file now carries an RDB version. An older Redis binary cannot read an RDB body written by a newer one, so downgrades are constrained in the same way snapshot files are. ## The Redis 7 layout Since **Redis 7.0** the AOF is no longer one file but a directory (`appenddirname`, default `appendonlydir`) holding a manifest, a **base** file, and one or more **incr** files. With the preamble on, the base file is named `*.base.rdb` and holds exactly the binary body described above; with it off, the base is `*.base.aof` in command form. The hybrid idea is identical — it is just that body and tail now live in separate files described by a manifest instead of being concatenated. ## How to check it on a running server `CONFIG GET aof-use-rdb-preamble` reports the setting; `INFO persistence` shows whether AOF is enabled and the current sizes. On disk, the first five bytes of the AOF (or of the base file on 7.x) being `REDIS` is the direct evidence that the body is a snapshot.

  • Does enabling the RDB preamble change how much data you can lose in a crash?
    No. The preamble only changes the encoding of the part of the dataset that existed at the last rewrite. Everything written after the rewrite is still an ordinary command tail flushed according to `appendfsync`, so the exposure window is unchanged — about one second with the default `everysec`. It is a recovery-time optimization, not a durability one.
  • How would you tell whether a given AOF on disk was written with the preamble enabled?
    Look at the first bytes of the file (or of the base file on Redis 7.x). A hybrid body starts with the ASCII magic `REDIS` followed by the RDB format version; a pure command AOF starts with a RESP array, usually a `SELECT` command. On 7.0+ the manifest and the base filename also give it away: `.base.rdb` means preamble on, `.base.aof` means off.
  • If the preamble is so much faster to load, why not just run snapshots alone?
    Because a snapshot alone loses every write since the last save point, which can be minutes or an hour. The hybrid AOF gives you snapshot-speed loading for the bulk of the data plus a command tail that can be fsynced every second, so the loss window stays around a second while recovery stays fast.

Like a savings account statement that opens with an itemized balance sheet as of the last statement date, then lists only the transactions since. You do not re-add every transaction since account opening to learn the balance.

saying these in an interview costs you the question

  • Claiming the preamble improves durability or replaces `appendfsync always`
  • Describing it as Redis merging dump.rdb and appendonly.aof at load time
  • Thinking it is off by default and must be enabled on modern Redis
  • Believing it removes the need for separate snapshot backups
  • Saying the whole AOF becomes binary — the tail after the last rewrite is still RESP text

context

open as a page

A Redis server is configured with both snapshotting and the append-only file enabled, and it restarts. Which of the two files does it use to rebuild the dataset, and what goes wrong if someone enables the append-only file by editing the config file and restarting?

level: juniorimportance: should knowfreq 40%

basics

~20 s

If appendonly yes, Redis loads the append-only file and ignores the snapshot, because the AOF is more current. Turning AOF on by editing the config and restarting loads an empty AOF, so the server comes up empty and can overwrite the snapshot.

open as a page

On Redis 7.x the append-only file is a directory (default `appendonlydir`) containing a `.manifest` file, a base file, and one or more `.incr.aof` files. As the operator: what is the unit you must copy or restore, how does the server decide what to load at startup, where do you point `redis-check-aof`, and what breaks if a script copies or truncates one file inside that directory on its own?

level: seniorimportance: should knowfreq 24%

basics

~20 s

The copy unit is the whole directory — manifest plus the base file plus every incr file it names, as one consistent set. Startup reads the manifest and loads base first, then incr files in manifest order. Point redis-check-aof at the manifest. A single file copied or truncated in isolation gives an unloadable or silently short dataset.

open as a page

You operate three Redis deployments: a page-fragment cache, a user session store, and a small ledger of business events that no other system holds. For each, decide whether to run snapshots only, the append-only file only, or both — and justify the configuration you would set.

level: principalimportance: should knowfreq 40%

basics

~20 s

Cache: snapshots only, or nothing — data is rebuildable. Sessions: both, with the append-only file on everysec, since losing minutes of logins hurts but is survivable. Ledger: both, with a real replica and off-box backups, because Redis alone should not be the system of record.

open as a page