What are write, read and space amplification in a log-structured engine, and why can tuning never minimise all three at once?

## Three ratios that describe an LSM engine A log-structured engine turns random writes into sequential ones, then pays for it later by merging. Three **amplification** ratios capture where the cost goes. | ratio | definition | main sources | |---|---|---| | **write amplification** | bytes written to storage ÷ bytes written by the application | commit log, flush, every compaction that rewrites the data | | **read amplification** | work per read ÷ the minimum needed (files, blocks, seeks) | overlapping files, versions and delete markers to merge | | **space amplification** | bytes on disk ÷ bytes of live data | superseded versions, delete markers, temporary space during compaction | ## Why they pull against each other The central decision is **how often and how aggressively to merge files**. - **Merge eagerly** (keep data in few, non-overlapping files): - reads touch few files — **low read amplification**; - obsolete data is dropped quickly — **low space amplification**; - but each byte is rewritten each time it moves into a larger file — **high write amplification**. - **Merge lazily** (let files accumulate, merge in big batches): - each byte is rewritten fewer times — **low write amplification**; - but reads must check many overlapping files — **high read amplification**; - and stale versions linger — **high space amplification**, with large temporary space needed when a big merge runs. Research on storage engines has formalised this: improving any two of read, update and memory or space overhead tends to worsen the third. No setting minimises all three at once. ## How workload shape moves the balance 1. **Append-only, never updated** (logs, events): little stale data exists, so lazy merging costs little space; write amplification dominates. 2. **Update- and delete-heavy**: stale versions accumulate, so lazy merging hurts reads and space badly; eager merging pays off. 3. **Time-ordered with expiry**: grouping files by time lets whole files expire without rewriting, cutting all three. 4. **Read-latency critical**: keep read amplification low even at write cost. The concrete compaction strategies that sit at different points of this trade-off, and how deletes and expiry interact with them, are a subject of their own. ## Measuring before tuning - **Write amplification**: compare disk bytes written (from the operating system or engine statistics) with bytes ingested. - **Read amplification**: engine metrics for files or blocks read per query, and read latency percentiles. - **Space amplification**: disk used versus an estimate of live data, plus headroom needed for the largest compaction. Watch them together: a change that improves read latency may quietly double disk writes, which shortens SSD life and eats I/O budget. ## Interview angle Define all three precisely, explain the eager-versus-lazy merge trade-off, tie it to workload shapes, and say how you would measure before changing anything.