With deletion vectors enabled on a Delta Lake table, what does a DELETE write?

## The problem Without deletion vectors, a Delta `DELETE`, `UPDATE` or `MERGE` is copy-on-write: any data file containing an affected row is rewritten in full, minus or with the change. Deleting three rows from a 256 MB Parquet file rewrites 256 MB and copies a quarter of a million untouched rows — visible in the commit's `numCopiedRows` metric. For GDPR-style scattered deletes, or for a `MERGE` touching a few rows in many files, this write amplification dominates the job. ## The mechanism Enabling the feature (`delta.enableDeletionVectors = true`) switches those operations to merge-on-read. The writer: 1. determines which **row positions** inside each affected data file are deleted — ordinal positions in the file, not primary keys; 2. serialises those positions as a compressed bitmap (a roaring-bitmap encoding) into a side file in the table directory, or inline in the log when tiny; 3. commits a `remove` action for the file's previous state and an `add` action for the **same `path`**, now carrying a `deletionVector` object. The `deletionVector` object names `storageType` (whether the bitmap lives in a separate file, at an absolute path, or inline), `pathOrInlineDv`, `offset` and `sizeInBytes` locating the bitmap, and `cardinality` — how many rows it marks deleted. That cardinality is what lets the engine report accurate row counts without opening the bitmap. No Parquet data is written. A delete becomes proportional to the number of *files touched*, not to the bytes they contain. ## What readers must do A scan of a file that has a deletion vector must fetch the bitmap and skip the marked row positions. That is a genuine change to how an `add` action is interpreted: previously, a live `add` meant every row in that file is part of the table. This is why `deletionVectors` is a **reader-writer** table feature, appearing in both `readerFeatures` and `writerFeatures`, and why the table's protocol jumps to reader version 3 / writer version 7. An engine that does not implement deletion vectors must refuse the table outright — the alternative would be silently returning rows the user deleted, which is a correctness and compliance failure, not a performance one. Enabling the feature on a table read by an older Spark, an older connector or a third-party tool will break those consumers, so inventory them first. ## Costs and accumulation Merge-on-read moves work rather than removing it: - **Read cost.** Every scan of a covered file loads and applies its bitmap. Cheap per file, but non-zero, and it constrains vectorised reading. - **Skew between physical and logical size.** A file may be 256 MB on disk while only a fraction of its rows are live. Enough deletes and the table is mostly reading rows it then throws away. - **Small side files.** Each round of deletes adds bitmap files. The remedy is compaction. `OPTIMIZE` rewrites affected files and, in doing so, materialises the deletions — the rewritten files contain only live rows and carry no vector. Where the physical bytes must genuinely go (a right-to-erasure request, for example), `REORG TABLE ... APPLY (PURGE)` forces the rewrite of files carrying deletion vectors, and the retention-based cleanup then removes the tombstoned originals once the retention window has passed. Until both steps run, the deleted rows still exist in the old Parquet files even though no query returns them, which is exactly the distinction a compliance reviewer will probe. ## Contrast with the neighbouring formats Every table format has some answer to "delete without rewriting", and mixing them up is the classic interview error. Delta's answer is the deletion vector: a positional bitmap attached to one data file. Iceberg's answer is delete files — positional deletes and equality deletes — which are separate data files tracked in its own metadata. Hudi's answer is a Merge-on-Read table type where updates land in log files that are later compacted into base files. Same problem, three different mechanisms, and the vocabulary does not transfer. ## Where it also helps Because a deletion vector rewrites no data file, two writers modifying different rows in the same file are less likely to be found in conflict; recent Databricks runtimes build row-level concurrency on top of deletion vectors and row tracking specifically to reduce the concurrent-modification failures that plague `MERGE` on unpartitioned tables. That is a runtime-level capability layered on the feature, not a property of the bitmap itself. ## When to leave it off Append-mostly tables with rare deletes gain little and still pay the protocol upgrade, which may lock out consumers. Tables read by a heterogeneous fleet of engines are the risky case. Delete-heavy or `MERGE`-heavy tables read by an up-to-date engine are where the feature pays for itself — provided compaction runs regularly, because merge-on-read without compaction degrades exactly the way it does in every other format.