skip to content

Why can a vector store usually move an index shard to another node by copying it, but not split one without rebuilding its index?

level: middleimportance: should knowfreq 40%

answer

  1. the shard as a sealed unit
  2. links that cross the cut
  3. segments change the answer
  4. create more shards than nodes

basics

~20 s

A moved shard keeps every vector, so its index stays valid and copies as files. A split strands graph links across the cut, so each half needs a new index, though segment-based stores can sometimes reassign whole segments instead.

solid answer

~50 s

A shard's index is built over exactly the vectors in that shard. **Moving** the shard keeps that set intact, so its files are copied to the new node and loaded unchanged. **Splitting** divides the vectors into two sets, and a graph index does not divide: its links connect vectors that now land on different halves, and the internal IDs, entry point and upper layers all assume the whole set. Dropping half the nodes from each copy leaves graphs full of dead links, so each half is rebuilt. Inverted-list indexes split more easily because each list can be divided, though the trained centroids may fit the halves poorly. Segment-based stores, where data sits in many small immutable segments each with its own index, can sometimes split by assigning whole segments. Teams avoid live splits by creating many more shards than nodes, so growth is only moves.

go deeper

for a junior

Recall that moving a shard copies it unchanged, while splitting a graph-indexed shard means building new indexes for the halves.

for a middle

Explain why graph links crossing the cut break navigability, and why inverted lists and segment-based storage split more gracefully.

for a senior

Plan growth as moves: shards per node, the cost of over-provisioning in fan-out, and an offline build-then-switch path for the split you cannot avoid.

for a principal

Choose the initial shard count as a deliberate trade between years of cheap growth and per-query fan-out today, and own the rebuild window it implies.

## A shard is a sealed unit In a vector store, a **shard** is a subset of the vectors plus an **approximate nearest-neighbour index** built over exactly that subset. The index's internals refer only to vectors inside the shard: a graph index stores links between them, using shard-local IDs, with an entry point and sparse upper layers chosen from the same set. That makes a shard easy to **move**. Moving changes where the shard lives, not what it contains, so every link still points at a vector that is present. The store copies the shard's files to the new node, loads them, catches up any writes made during the copy and switches routing. Nothing inside the index changes. ## Why a split breaks a graph index **Splitting** a shard means dividing its vectors into two new shards, by key range, by hash or by cluster. The graph does not divide along that line: - **Links cross the cut.** A vector's neighbours were chosen from the whole shard. After a split, many of them live on the other half, so each half's graph has links pointing at vectors it no longer holds. - **Navigability depends on those links.** Graph search walks from neighbour to neighbour. Remove a large fraction of nodes and the walk hits dead ends, recall falls and latency rises. - **Global structures are lost.** The entry point and upper layers sit on one half; the other half has none. - **Local IDs need renumbering**, which touches every link anyway. A half can be patched by treating the other half's vectors as deleted, but that is a graph carrying 50% tombstones: memory is not freed, search wastes hops on dead nodes, and recall degrades the way it does under heavy deletes (covered with how deletes affect a graph index). In practice each half gets a **fresh index built over its own vectors**. ## Other index types and segment-based stores The answer is hedged because not every store and index behaves the same way: | Index or storage shape | Move | Split | |---|---|---| | One graph per shard | Copy files | Rebuild each half's graph | | Inverted lists over trained centroids | Copy files | Divide each list; centroids may fit the halves poorly and need retraining | | Many immutable segments per shard, each with its own small index | Copy segments | Often reassign whole segments; segments that mix both halves are rewritten | | Flat (exhaustive) storage | Copy files | Divide the vectors; nothing to rebuild | In **segment-based** stores a shard is a collection of segments, each holding a batch of vectors and its own small index. If the split key lines up with segments, a split can hand whole segments to each side with no rebuild at that moment. Segments holding vectors from both sides still have to be rewritten, and background merges later build larger indexes, so rebuilding is deferred and spread out rather than avoided. ## Designing so growth is a move Because moves are cheap and splits are not, teams arrange for growth to need only moves: 1. **Create more shards than nodes at the start.** A scenario's cluster of 8 nodes with 64 shards holds 8 shards per node. Growing to 16 nodes means 4 shards per node, so each original node hands 4 shards to a new one, with no rebuild anywhere. 2. **Size shards so a rebuild fits a maintenance window**, for the day a split or re-routing is unavoidable. 3. **When a split is unavoidable, build offline.** Build the two new shards from kept vectors while the old shard serves, replay writes made meanwhile, verify, then switch routing in one step. Over-provisioning shards is not free. Under random placement every query visits every shard, so 64 shards means 64 searches per query even when 8 would hold the data. The shard count is a trade between cheap growth and per-query fan-out. ## Common mistakes - Treating a split like cutting a file in half. - Assuming a move needs an index rebuild on the destination. - Thinking equal halves make a split cheap: balance does not reconnect links. - Pre-creating thousands of tiny shards and paying for the fan-out on every query.

  • Why not split a graph-indexed shard by copying it twice and marking half the vectors deleted on each side?
    Each side then carries a graph that is half tombstones. Memory is not freed, searches waste hops on dead nodes, and recall falls as links to live nodes thin out, so the store ends up rebuilding at its next compaction anyway. It can be a stopgap, not a split.
  • How do teams avoid live shard splits altogether?
    They create many more shards than nodes at the start, so adding nodes only moves whole shards. When a split cannot be avoided, they build the two new shards offline from kept vectors while the old one serves, replay the writes made meanwhile, verify, and switch routing in one step.

A shard's graph index is like a city's street map. You can move the whole city's map to a new office unchanged, but cut the city in two and every street that crossed the cut now ends nowhere, so each half needs a new map drawn.

  • Splitting a graph-indexed shard just means cutting its index file in half.
  • Moving a shard requires rebuilding its index on the destination node.
  • A split is cheap as long as the two halves hold equal vector counts.
  • Pre-creating many small shards has no cost at query time.
  • Every store must fully rebuild on a split, segment-based stores included.