Before Redis 7.0, a rewrite of the append-only file required buffering writes that arrived while the child worked and appending them at the end. What did Redis 7.0's multi-part AOF change, and which problems did it solve?

## The pre-7.0 design and its pain points Up to Redis 6.x, AOF persistence used a single file, `appendonly.aof`, in the working directory. Rewriting it worked like this: 1. Fork a child. The child serializes the fork-time dataset into a temporary file. 2. The parent keeps serving. Every write it executes goes to two places: the current AOF (for durability if the rewrite fails) and an in-memory **AOF rewrite buffer**. 3. The parent continuously ships the buffer's contents to the child through a pipe, so the child can append them to its temp file as it goes. 4. When the child finishes, the parent appends whatever diff remains and `rename()`s the temp file over `appendonly.aof`. The correctness of this is fine; the cost is not. - **Memory**: the rewrite buffer is unbounded in the sense that it holds every write for the duration of the rewrite. A long rewrite on a write-heavy instance could add gigabytes of RSS *on top of* copy-on-write, which is precisely the moment you have the least memory headroom. - **CPU and write amplification**: every command was written to the AOF, copied into the buffer, piped to the child, and written again into the new file. - **A tail burst**: the final diff-append was one large write right as the child completed, a bad time for latency. - **All-or-nothing**: the entire dataset had to be re-serialized into one file each time. ## What 7.0 changed Redis 7.0 replaced the single file with a **multi-part AOF** living in its own directory: ``` appenddirname "appendonlydir" appendonlydir/ appendonly.aof.1.base.rdb # base: dataset image at rewrite time appendonly.aof.1.incr.aof # incremental: commands since that base appendonly.aof.manifest # which files, in which order, and their types ``` Three file kinds: - **BASE** — the dataset as of the last rewrite. In RDB format when `aof-use-rdb-preamble yes` (the default), otherwise as AOF commands. - **INCR** — commands appended since that base. There is normally one active INCR file, occasionally more, for example during a failed or interrupted rewrite. - **MANIFEST** — a small text file naming the parts and their sequence. It is the authority on what constitutes the current AOF; a part file present on disk but absent from the manifest is not part of the dataset. The rewrite now works differently in exactly the place that used to hurt: 1. The parent opens a **new INCR file** and starts appending concurrent writes to it directly. No rewrite buffer, no pipe. 2. The child writes a new BASE from the fork-time snapshot. 3. On success, Redis writes a new manifest naming the new base plus the new incr file, atomically replaces the manifest, and unlinks the obsolete parts. Because the parent's concurrent writes were already durable in a real file, there is nothing to append at the end, and nothing held in memory. ## What this buys you - **Lower memory during rewrites**: the rewrite buffer is gone. `INFO persistence` no longer reports `aof_rewrite_buffer_length` in 7.x, which is itself a useful version tell. - **Less write amplification and CPU**: each command is written once to the incr file, not written, buffered, piped and rewritten. - **No completion burst**: the swap is a manifest update, not a large append. - **Cleaner failure handling**: an interrupted rewrite leaves a stale base or incr file on disk that the manifest simply does not reference. The previous manifest remains valid, so the dataset is intact and the debris can be cleaned up. ## What it costs you operationally The things people trip over are all about the file layout changing: - **Backups**: copying `appendonly.aof` no longer means anything. A consistent AOF backup is now the whole directory copied together with its manifest; parts copied without a matching manifest are unusable. This is one more reason most teams back up an RDB instead. - **Repair tooling**: `redis-check-aof` in 7.x is pointed at the manifest, and it understands the multi-part layout. - **Upgrade path**: starting 7.x against an old single-file `appendonly.aof` is handled, and the layout is migrated to the directory form; downgrading afterwards is not something to improvise, since a 6.x server does not understand a manifest. - **Monitoring and scripts** that grepped for a single filename, or watched its size, need updating: current size is now the sum of the parts named by the manifest. ## Related 7.0 detail worth knowing Redis 7.0 also added optional **timestamp annotations** in the AOF (`aof-timestamp-enabled`, off by default), which write periodic timestamp comments into the log. That is what makes `redis-check-aof --truncate-to-timestamp` possible, giving a crude point-in-time recovery within a single AOF. It is a separate feature from the multi-part layout but shipped in the same release and often comes up in the same conversation.