skip to content

You keep 90 days of event history in a very large table and must expire the oldest day every night. Compare running a DELETE for the expired rows against dropping a whole time-based partition: what does each actually cost the database, and what does the cheap option require of the table's design?

level: middleimportance: must knowfreq 68%

answer

  1. DELETE = dead tuples + WAL per row + vacuum debt
  2. DROP partition = catalog + unlink, O(1)
  3. space returned only by drop/truncate/rewrite
  4. only works if partition key == retention key
  5. detach, archive, then drop

basics

~20 s

DELETE touches every row: a log record per row, dead row versions, index entries left behind, bloat, and vacuum work afterwards, and the space is not returned. Dropping a partition is a catalog change plus file unlink: constant time, space freed at commit. It requires partitioning by the retention key.

solid answer

~60 s

For bulk retention, DELETE is the expensive option. Under MVCC a delete does not remove bytes: it marks each row version dead, writes a write-ahead-log record per row, dirties every page it touches, and leaves index entries pointing at dead tuples. Space is reclaimed later by vacuum/purge and normally not returned to the filesystem, so the table stays bloated and replicas replay all that log volume. Dropping a partition is a metadata operation: update the catalog, drop that partition's own indexes, unlink its files. It is O(1) in the number of rows, produces a handful of log records, and returns disk space at commit. The precondition is design: the table must be partitioned on the retention key, at a granularity where one retention step equals a whole number of partitions (daily partitions for a daily rolling window). If you partition by tenant but expire by date, you are back to row-by-row deletion. Operationally the drop needs a brief exclusive lock on the parent, so use a lock timeout and retry instead of queueing behind long-running readers.

code

sql · 11 lines
sql
CREATE TABLE events (
  id          bigint       NOT NULL,
  occurred_at timestamptz  NOT NULL,
  payload     jsonb
) PARTITION BY RANGE (occurred_at);

CREATE TABLE events_2026_05_04 PARTITION OF events
  FOR VALUES FROM ('2026-05-04') TO ('2026-05-05');

-- nightly retention step
DROP TABLE events_2026_05_04;

go deeper

for a junior

Know the headline: deleting millions of rows is slow and leaves the table bloated, dropping a partition is near-instant because it just removes a file, and this only works if you partitioned by date.

for a middle

Explain the MVCC mechanics: dead tuples, WAL per row, index entries, vacuum afterwards, space not returned; and state the design precondition that partition granularity must match the retention step.

for a senior

Add the operational side: lock behaviour of the DDL, lock_timeout and retry, replication lag from WAL volume, detach-then-archive, and when row-level deletes are still unavoidable.

for a principal

Frame retention as a driver of the partitioning decision itself: granularity versus partition count, per-tenant policies, archival tiering, and the cost model of the maintenance job across a fleet of tables.

## The job being described A rolling-window retention policy ("keep the last 90 days") is one of the heaviest recurring workloads on a high-volume table, because it repeatedly touches the largest, coldest part of the data. How you implement it is largely decided when you choose the partitioning scheme, not when you write the job. ## What a DELETE actually does In a multi-version concurrency control (MVCC) engine, rows are never overwritten in place while other transactions may still need them. A DELETE therefore: - marks each row version dead (in PostgreSQL, stamps the deleting transaction id) rather than freeing space; - writes a write-ahead-log (WAL) record per row, which is shipped to replicas and to the archive; - dirties every heap page it touches, so those pages must be written back, evicting hot pages from the buffer cache; - leaves every secondary index entry in place, to be cleaned later; - fires row triggers and foreign-key checks per row; - creates vacuum/purge debt: PostgreSQL autovacuum must scan the table and all its indexes to reclaim the dead tuples, InnoDB accumulates undo and purge work. Afterwards the file is still the same size. Free space is reusable by future inserts, but returning it to the operating system needs a rewrite (VACUUM FULL, pg_repack, OPTIMIZE TABLE) that takes a heavy lock. A single 200-million-row delete can also blow up replication lag and, if run as one transaction, hold locks and pin the oldest transaction horizon so that nothing else can be vacuumed either. ## What dropping a partition does If the oldest day lives in its own partition, expiry becomes a DDL statement: remove the partition's catalog entries, drop the indexes that belong to that partition, and unlink its data files at commit. The cost does not depend on how many rows are inside. There is no per-row logging, no index cleanup, no bloat, no vacuum debt, and the disk space comes back immediately. The same trick exists in every partitioning implementation: ALTER TABLE ... DROP PARTITION in Oracle and MySQL, DROP TABLE of the child in PostgreSQL declarative partitioning. TRUNCATE of a partition is equally cheap if you want to keep the empty partition attached. ## The design precondition Dropping only works when the retention predicate lines up with partition boundaries: - partition on the column the policy is expressed in (event timestamp, not tenant id); - pick a granularity so that one expiry step removes whole partitions: daily partitions for a daily window, monthly for a monthly window; - resist mixing policies ("90 days for tenant A, one year for tenant B") unless the partition key can express them, otherwise the long tail still needs deletes. A useful sanity check in an interview: state that partitioning is often adopted for exactly this reason, retention, more than for query speed. ## Locking and safety Dropping or detaching a partition needs an exclusive lock on the parent table. That lock must wait for existing transactions touching the parent, and while it waits, every new query queues behind it, so a single long-running report can turn a millisecond DDL into a site-wide stall. Mitigations: set a short lock_timeout and retry with backoff, run in a low-traffic window, and prefer a concurrent detach (available in newer PostgreSQL) followed by dropping the now-standalone table. ## Archiving instead of discarding If the data must be kept: detach the partition so it becomes an ordinary table, export or move it (COPY to object storage, dump, move to cheaper storage), then drop it. Detach-then-archive keeps the exclusive-lock window short and the export off the live parent. ## When DELETE is still the right tool Row-level erasure that does not align with partitions (deleting one user's data on request), small volumes, or tables that are not partitioned. Then: delete in bounded batches keyed by the primary key, commit each batch, throttle to let vacuum and replication keep up, and expect to schedule a rebuild if the table becomes badly bloated.

  • Dropping a partition needs an exclusive lock on the parent. Why can that be dangerous on a busy system, and what do you do about it?
    The DDL waits for every transaction currently touching the parent table, and while it waits in the lock queue all new queries queue behind it, so one slow report can stall the whole table. Set a short lock_timeout so the statement fails fast, retry with backoff, and run the job in a quiet window. Where available, detach the partition concurrently first and then drop the standalone table.
  • The business now wants the expired data archived, not destroyed. How does the procedure change?
    Detach the partition instead of dropping it; it becomes an ordinary standalone table that no longer participates in queries against the parent. Export it at leisure (COPY to files or object storage, or move it to a cheaper tablespace or an archival system), then drop the standalone table. This keeps the exclusive-lock window to the detach itself rather than holding it for the whole export.

Deleting rows is erasing a filing cabinet page by page and leaving the empty folders in place; dropping a partition is throwing out the whole drawer.

saying these in an interview costs you the question

  • Claiming DELETE returns disk space to the operating system immediately.
  • Thinking batching the DELETE makes it as cheap as a drop; batching only limits lock and bloat spikes, the per-row work is unchanged.
  • Assuming any partitioning scheme enables cheap retention, without noticing the partition key must be the retention key.
  • Ignoring that the drop still needs a brief exclusive lock and can be blocked by long-running queries.

context