What does compacting a write log replace it with, what triggers it, and what does the compaction itself cost while the store serves?

## Why the log outgrows the data it describes The write log is a history of operations, not a picture of the keyspace. That has a direct consequence: **its size tracks write volume since it was last shortened, not how much data is live.** A single counter incremented a million times contributes a million entries and one live value. Entries whose keys were later removed are still in the file, as is every superseded value. A tier holding a modest working set can therefore carry a log many times larger than the keyspace it would rebuild. Two costs grow with it: - **Disk.** The file grows monotonically as long as writes arrive. - **Recovery time.** Replay at start applies every entry in the log, so a long history means a long start, even though the keyspace it ends up with is small. ## What compaction produces **Compacting the write log means replacing an accumulated log with a shorter one that reconstructs the same state.** The end state is identical; what disappears is the history of how it was reached. How stores produce that shorter file differs: - Some walk the live keyspace and emit a minimal set of operations that would recreate it from empty. - Some merge the existing log, collapsing repeated writes to the same key. - Some write a compact representation of the keyspace itself and then continue appending new operations after it. The floor is set by the live keyspace: a compacted log has to describe every value that is still there, so it cannot be smaller than that, however much history it discards. ## What triggers it Stores differ here, and the honest answer names the class of trigger rather than a number: - **A growth ratio** - the log has grown by some multiple of its size after the previous compaction. - **An absolute size** - the file crosses a configured threshold. - **A schedule**, run during a window with headroom. - **An operator asking for it**, typically before or after something else expensive. - **Continuous background maintenance**, in stores that fold compaction into ordinary housekeeping rather than treating it as an event. In several stores this is automatic, with a knob rather than a command; assuming an operator must trigger it is as wrong as assuming it never needs attention. ## What it costs while the store is serving Compaction runs against a process that is still accepting writes, which is the whole source of its cost. | Cost | What drives it | How it shows up | |---|---|---| | Disk | The live keyspace, plus writes arriving during the rewrite | Two logs on disk at once until the switch-over | | Memory | The delta of writes arriving while it runs - write rate times duration | Resident size rises for the duration, sometimes sharply | | Processor and disk bandwidth | Keyspace size and device throughput | A latency bump for concurrent callers | | A pause | The switch-over, and in some designs the buffered delta being applied | A brief stall rather than a sustained one | The writes arriving during the rewrite have to end up in the new log. The store either buffers them and appends them after the switch-over, or writes them to both files as it goes. Either way the delta is proportional to **write rate times how long the compaction takes**, which is why the same compaction is cheap at night and expensive at peak. One failure mode is worth naming: if compaction cannot finish faster than writes accumulate, the log grows anyway and the work has been spent for nothing. That is a signal the tier is being asked to absorb a write rate this posture cannot pay for. ## Operating around it 1. **Leave headroom in memory** for the delta, not just for the keyspace, and leave disk for two logs at once. 2. **Do not set the trigger so tight** that compaction runs continuously; a log that is allowed to grow somewhat before being shortened costs less overall than one rewritten constantly. 3. **Watch when it overlaps other expensive work** - a compaction competing with a traffic peak is a latency incident with an obvious cause and a non-obvious signature. 4. **Treat it as the lever on recovery time**, not only on disk usage. The reason to keep the log short is usually how long a start would take, and that connection is what a weak answer misses.