skip to content

In a columnar table whose data files are immutable, how does an UPDATE or DELETE actually work?

level: middleimportance: must knowfreq 75%

answer

  1. files are sealed once written
  2. the unit of replacement is a file
  3. cost tracks files touched, not rows matched
  4. metadata swap makes the change atomic
  5. old files linger for the retention window

basics

~20 s

The engine never edits a file. It finds the files containing affected rows, writes new files holding the surviving and modified rows, and atomically swaps the file set in the table metadata. Old files stay until retention expires, then are dropped.

solid answer

~50 s

Mutation is **copy-on-write at file granularity**. The engine resolves which immutable files contain rows matching the predicate, reads each of them, writes out replacement files containing the unaffected rows plus the new values for the changed ones, and then commits a metadata change that atomically swaps the old file list for the new one. Readers already running keep seeing the old set, which is how snapshot isolation and time travel come for free. Two consequences dominate interviews. First, cost scales with **files touched, not rows changed** — deleting one row from a 400 MB file rewrites all 400 MB. Second, a predicate that is uncorrelated with the table's physical layout touches every file, so `DELETE WHERE user_id = 7` on a table laid out by time can rewrite the whole table. Some engines soften this with merge-on-read markers instead of immediate rewrites, deferring the rewrite to a later compaction.

code

sql · 5 lines
sql
-- cheap: predicate matches the physical layout, few files rewritten
DELETE FROM events WHERE event_date = DATE '2026-01-05';

-- expensive: one matching row per file rewrites nearly every file
DELETE FROM events WHERE user_id = 7;

go deeper

for a junior

Recall that data files are never edited: a change means writing new files and pointing the table at them. Know that deleting a single row is far more expensive here than in a transactional database.

for a middle

Explain copy-on-write end to end — candidate files, rewrite, atomic metadata swap, deferred cleanup — and state clearly that cost scales with files touched rather than rows changed.

for a senior

Show judgment about mutation patterns in production: batching scattered deletes, aligning predicates with the physical layout, choosing a rebuild over an in-place statement, and expecting the temporary storage spike.

for a principal

Own the design question of whether the table should be mutable at all. Weigh append-only plus late reconciliation against in-place mutation, and account for compliance deletes, retention windows and their cost over the platform's lifetime.

## Copy-on-write, not in-place edit Analytical storage is written once and never modified. There is no equivalent of an in-place page update, no free space reserved inside a file, no row-level lock on a byte range. So a statement that logically changes rows is implemented as a **rewrite**: 1. The planner resolves which files can contain matching rows, using each file's per-column statistics to skip the ones that provably cannot. 2. Each candidate file is read. 3. New files are written containing the rows that survive — unchanged rows copied verbatim, changed rows with new values, deleted rows omitted. 4. A metadata commit atomically replaces the old file identifiers with the new ones in the table's current file set. 5. The superseded files are left in place until a retention window expires, then garbage-collected. Step 4 is why the operation is atomic and why concurrent readers never see a half-applied change: a reader pinned the previous file set when it started and keeps reading it. Step 5 is why an analytical table's storage bill can be several times its logical size right after a large mutation, and why time travel and rollback are natural in these systems rather than a bolted-on feature. ## Cost scales with files touched, not rows changed This is the single most important consequence, and the thing candidates most often get wrong. Deleting one row from a table stored as 400 MB files does not do 100 bytes of work — it reads and rewrites the entire file that held the row. Updating one column of one row does the same, because the file is the unit of replacement, not the column and not the row. The multiplier is how many files the predicate reaches: ```sql -- table physically ordered by event_date DELETE FROM events WHERE event_date = DATE '2026-01-05'; -- touches a few files DELETE FROM events WHERE user_id = 7; -- may touch every file ``` The first predicate is correlated with the physical layout, so the file statistics eliminate almost everything and the rewrite is proportional to one day of data. The second is scattered across the whole table: one matching row per file means every file is rewritten, and a delete of a thousand rows can rewrite terabytes. Right-to-be-forgotten deletions are the classic production version of this problem, which is why teams often batch such deletes into a periodic job rather than executing them one subject at a time. ## Merge-on-read as the alternative Some engines and table formats avoid the immediate rewrite by recording that certain rows are logically gone or superseded — a marker file, a tombstone row, or a bitmap of deleted row positions — and applying it when the data is read. The write becomes tiny; the read pays a merge cost until a later compaction materialises the change. Engines built around continuously merged parts take a related approach: a mutation is expressed as a new part that wins over older rows, and the reconciliation happens during background merges or when the reader is explicitly asked to reconcile. The trade is symmetric and worth stating plainly: rewriting the file up front costs write amplification once and keeps reads clean; deferring costs almost nothing on write and taxes every read until compaction runs. ## Practical guidance - **Prefer set-based mutations.** One statement that deletes a month is far cheaper than thirty statements deleting a day each, because files are rewritten once instead of repeatedly. - **Align mutation predicates with the physical layout** wherever the workload allows, so file-level statistics can eliminate most files. - **Batch scattered deletes.** Accumulate the keys and run one job on a schedule rather than mutating on every request. - **Consider a rebuild.** When a mutation would touch most of the table anyway, writing a fresh table from a filtered query is often cheaper and easier to reason about than an in-place statement. - **Watch storage after big mutations.** Superseded files persist for the retention window; if your table doubled in size overnight, that is usually why. ## What interviewers are listening for The give-away weak answer is "the engine updates the row like any database, it is just slower". The strong answer names copy-on-write, ties cost to files touched rather than rows changed, explains the atomic metadata swap that gives snapshot isolation and time travel, and mentions the merge-on-read alternative and its read-side cost.

  • Why does a table's storage footprint grow after a large DELETE instead of shrinking?
    The delete writes new replacement files while the superseded files remain readable for the retention or time-travel window. Until that window expires and garbage collection runs, the table holds both versions. Storage only drops afterwards, so a delete temporarily increases cost rather than reducing it.
  • How does this mutation model give you snapshot isolation almost for free?
    A reader pins the file set that was current when it started, and files are never modified in place, so a concurrent mutation cannot change what that reader sees. The mutation's only visible act is the atomic metadata swap, which affects readers that start afterwards. Isolation falls out of immutability plus a single-point metadata commit.
  • When would you rebuild the table instead of issuing an UPDATE?
    When the predicate is uncorrelated with the layout and would touch most files anyway. Writing `CREATE TABLE new AS SELECT ...` with the transformation applied does one pass, produces well-sized files, and lets you re-sort the data at the same time — often cheaper than an in-place statement that rewrites the same bytes with worse file sizing.
  • Why are many small mutations worse than one large one covering the same rows?
    Each statement rewrites the files it touches, so overlapping statements rewrite the same file repeatedly — pure write amplification. Each also produces its own commit and its own set of surviving files, which tend to be smaller and fragment the table. One set-based statement reads and writes each affected file exactly once.

Editing a row is like correcting a typo on a printed page in a bound book: you cannot scratch it out, you reprint the whole page and re-bind, and the old page stays in the recycling bin for a while.

saying these in an interview costs you the question

  • Says the engine edits rows in place like an OLTP heap
  • Assumes deleting one row costs one row of work
  • Expects storage to shrink immediately after a DELETE
  • Thinks row-level locks protect concurrent readers here
  • Ignores that a scattered predicate rewrites every file

context