skip to content

When loading tens of millions of rows into an existing table, teams often drop the table's indexes first and rebuild them after the load. Why is that faster, and when would it be the wrong choice?

level: middleimportance: should knowfreq 38%

answer

  1. per-row: random page per index; bulk: sort once, write sequentially
  2. rebuild processes the WHOLE table, not just new rows
  3. new/existing ratio decides it
  4. unique index = constraint, not just speed
  5. alternatives: presort input, load into a new partition, swap

basics

~20 s

Maintaining indexes row by row means a random page touch per index per row, with splits and logging. Building an index once afterwards reads the data in bulk, sorts it, and writes packed leaf pages sequentially. It is wrong when the table stays online, when constraints must hold during the load, or when the existing data dwarfs the new rows.

solid answer

~1 min

**Why it is faster.** Incremental maintenance costs, per row, a tree descent plus a modification on a page chosen by the key value — effectively random I/O — in every index, plus page splits and log records. A bulk build instead scans the table once, sorts the keys (using large sort buffers and sequential I/O), and writes dense leaf pages in order, bottom-up. Sorting a large set once is far cheaper than millions of independent random insertions, and the resulting index is compact with no split-induced fragmentation. **When it is wrong:** - **The table is live.** Without the indexes, concurrent queries fall back to full scans, and unique/foreign-key constraints backed by those indexes stop being enforced during the window. - **The load is small relative to the table.** Adding 5M rows to a 2B-row table means rebuilding indexes over 2.005B rows — far more work than maintaining 5M insertions. - **The rebuild window is unacceptable**, or there is not enough temporary space and memory for the sort. - **Recovery risk**: if the load fails mid-way, you are left with a table that has no indexes and needs a long rebuild before it can serve traffic again. Middle ground: keep the primary key and any constraint-critical unique index, drop only the non-constraint secondary ones, and use a concurrent/online rebuild if the engine supports it.

go deeper

for a junior

Explain that maintaining indexes row by row is slow and that building an index once from sorted data is faster, so bulk loads often drop indexes first.

for a middle

Add the sort-and-build mechanism versus random per-row insertion, and name the main caveats: availability, constraint enforcement, and load size relative to the table.

for a senior

Quantify the ratio decision, cover online index builds, partition-attach and table-swap patterns, failure handling, and post-load statistics refresh.

for a principal

Treat it as a loading architecture decision: staging plus atomic swap versus in-place load, what availability guarantee the table owes, and how the ingestion pipeline is designed so this question stops recurring.

## Two ways to get an index built **Incremental maintenance** — the index exists, and each inserted row adds an entry to it. Per row, per index: compute the key, descend the tree, read the target leaf page (possibly a physical read), insert in sorted position, mark dirty, log it, and split the page if it is full. The target page is determined by the key, so for anything but a monotonically increasing key, successive rows hit unrelated pages. Ten million rows across four indexes is forty million such operations, scattered across the whole address space of each index. **Bulk build** — the index does not exist during the load; afterwards the engine reads the table once (sequentially), extracts the keys, sorts them using a large work area and sequential spill files, then writes the leaf level in order and constructs the upper levels on top. Sorting is the dominant cost and it is I/O-efficient. Nothing splits, because pages are filled to the intended fill factor as they are created, and the tree is built once with the right shape. The difference is the same as the difference between inserting into a sorted array one element at a time and sorting the array once at the end. For large volumes the bulk path routinely wins by a large factor, and it also produces a *better* index: densely packed, physically ordered, with no fragmentation from mid-page splits. ## The full cost accounting Dropping and rebuilding also avoids: - Log volume for millions of individual index modifications. - Buffer-cache thrash as index pages for four different indexes compete during the load. - The garbage left behind if the load includes updates or upserts. But it introduces its own costs: - The rebuild itself is a large, resource-hungry operation needing temporary space for sorting and significant memory. - The table is unindexed for the duration of load plus rebuild. - Rebuilding is all-or-nothing work over the *entire* table, not just the newly loaded rows. ## When the tradeoff flips **Ratio of new rows to existing rows.** This is the decisive number. Loading 40M rows into an empty or small table: drop and rebuild, easily. Loading 40M into a table already holding 3B: the rebuild processes 3.04B rows to save maintenance on 40M — a clear loss. A rough heuristic: the strategy pays when the new data is a substantial fraction of the total, and stops paying when it is a small percentage. **Availability requirements.** If the table serves live traffic, dropping indexes is usually off the table — queries collapse to scans and the system may not survive it. Engines that support building an index without blocking writes change this calculus: the build is slower and takes more resources, but the table stays usable. **Constraints.** A unique index is not just an accelerator; it is the enforcement mechanism for uniqueness. Dropping it means duplicates can enter during the load and the rebuild will then fail at the end — after hours of work — leaving you to find and remove the duplicates. Foreign keys referencing the table have similar dependencies. The safe pattern is: keep constraint-bearing indexes, drop only the pure-performance secondary ones. **Failure modes.** A load that dies halfway leaves a partially populated, completely unindexed table. Whether that is acceptable depends on whether the target is a live table or a staging one. Loading into a fresh staging table and then swapping it in (via a rename or a partition attach) sidesteps almost every one of these problems and is the pattern of choice when the schema allows it. ## Related levers worth mentioning - **Sort the input by the leading index key** before loading. If rows arrive in index order, even incremental maintenance becomes near-sequential: entries append to the same hot leaf page, splits are clean right-edge splits, and cache behaviour is excellent. This can capture much of the benefit without dropping anything. - **Batch commits** rather than one transaction per row, to amortise log flushes. - **Load into a new partition** and attach it, so the live table's existing indexes are never touched and only the new partition's indexes are built. - **Refresh statistics after the load**, always — a freshly loaded table with month-old statistics will produce bad plans regardless of how well the indexes were built. ## The interview-grade answer Explain the random-insert versus sort-and-build mechanism, then immediately qualify it with the ratio question and the availability/constraint constraints. A candidate who says "always drop indexes before a big load" has learned a rule without its boundary conditions; a candidate who asks "how big is the load relative to the table, and is the table serving traffic?" is thinking correctly.

  • How would you decide the cutoff between the two strategies in practice?
    Estimate the ratio of new rows to existing rows and the number of secondary indexes, then test both paths on a representative copy. If the load is a large fraction of the final table, drop-and-rebuild usually wins; if it is a small percentage, incremental maintenance wins because the rebuild would reprocess all the existing data. Availability and constraint requirements can veto the choice regardless of the numbers.
  • What is a lower-risk alternative that keeps the table available throughout?
    Load into a separate table or a new partition that carries its own indexes, build those indexes offline where nothing is reading them, then attach the partition or swap the tables in a single metadata operation. The live table's existing indexes are never dropped, there is no window without constraint enforcement, and a failed load can simply be discarded.
  • Why does sorting the input data by the index key help even if you keep the indexes in place?
    Because index maintenance cost is dominated by page locality, not by entry count. Rows arriving in key order all belong on the same rightmost leaf page, so that page stays cached, splits happen cleanly at the right edge, and there are few cold page reads. The same volume inserted in random key order scatters across the whole tree.

Filing a thousand new documents one at a time into five sorted cabinets means walking to a different drawer for each. Emptying the cabinets, sorting the whole stack once, and refiling in order is far less walking — unless the cabinets already hold a million documents you would have to re-sort too.

saying these in an interview costs you the question

  • Treating drop-and-rebuild as a universal rule without considering the size of the load relative to the table.
  • Dropping unique indexes and not realising uniqueness stops being enforced during the load.
  • Forgetting that a rebuild reprocesses the entire table, not just the newly loaded rows.
  • Not planning for the failure case where the load aborts and the table is left with no indexes.
  • Skipping the statistics refresh after a bulk load, then blaming the indexes for the slow queries that follow.

context