skip to content

Operating Vector Stores

Once the index is chosen, a vector store still has to run embedded, on a server or managed, shard and replicate, recover from a loss and make writes searchable. System-design loops probe each.

on this pageshow

questions

24

When is a vector index held in your web application's own memory enough, and what breaks once the app runs as several replicas?

level: juniorimportance: must knowfreq 55%

answer

  1. one process, one copy
  2. small, read-mostly corpora
  3. each replica builds its own copy
  4. a write lands on one replica
  5. restarts and scale-ups start empty

basics

~20 s

An in-process vector index is enough for a small, read-mostly corpus served by one process that can rebuild it at startup. With several replicas each holds its own copy, so a write reaches one copy, results diverge and every restart reloads.

solid answer

~50 s

An in-process index is a library-level index living in the application's own memory: no network hop and nothing extra to run. It is fine when the corpus is small, changes rarely, and can be rebuilt or loaded from a prebuilt file when the process starts. Run four replicas behind a load balancer and you have four independent indexes. A document added through one replica is searchable only there, so the same query answers differently depending on routing; every replica pays the memory again; and each restart, deploy or autoscaled instance must rebuild or reload before it serves a complete corpus. Durability is nobody's job either: anything written only to the index since the last load dies with the process. The way out is to move the index somewhere every replica shares, or to treat it as a read-only artifact built offline and shipped with each deploy.

go deeper

for a junior

Know what an in-process index is and the two things that make it fine: a small corpus and one process. Be ready to say that each replica gets its own private copy.

for a middle

Walk through what happens when a write lands on one of four replicas, and why restarts and autoscaled instances start empty. Mention that memory is paid once per replica.

for a senior

Show you can tell a legitimate in-process design, such as a per-release read-only artifact, from a prototype that outgrew itself, and explain why broadcasting writes is a replication protocol in disguise.

for a principal

Frame the move out of process as choosing who owns durability and concurrency, and set the trigger for it: live user writes or a replica count that makes the memory bill matter.

## What "in-process" means An **in-process vector index** is an approximate nearest-neighbour index (or a plain list of vectors searched exhaustively) built by a library inside the application's own process and held in its memory. There is no separate server: a search is a function call, with no network hop, no connection pool and no extra system to deploy, monitor or patch. It is usually the first shape a prototype takes, because a retrieval feature can be demonstrated in an afternoon. What the shape does not have is just as important. Nothing outside the process knows the index exists. Nothing persists it unless the application writes it out. Nothing coordinates it with any other copy. ## When it is enough The shape is a legitimate production choice, not only a prototype's, when all of these hold: - **One serving process**, or several that never need to agree with each other within a release. - **A small corpus** that fits comfortably in each process's memory with room to spare. - **Read-mostly data** that changes on a known cadence (nightly, per release), not continuously from users. - **A cheap rebuild**: the vectors are kept somewhere durable, so rebuilding the index at startup takes seconds, not hours. - **Tolerance for staleness** between rebuilds. A product catalogue or a documentation set that changes with each release, rebuilt in a build job and loaded at startup, meets every point. ## What breaks with several replicas Scale the web tier to four replicas and the same code now creates four indexes that share nothing. The failures arrive in a predictable order: 1. **Writes diverge.** A user uploads a document; the request lands on replica two, which adds its vectors to its own index. The other three never hear about it, so the same query returns different results depending on which replica the load balancer picks. 2. **Memory multiplies.** Each replica holds a full copy, so the index's memory cost is paid four times, and the replica count is now limited by index size as well as by traffic. 3. **Cold starts get slow.** Every deploy, crash restart and autoscaled new instance starts empty and must rebuild or reload before it can answer correctly. During that window it either refuses traffic or serves an incomplete corpus. 4. **Durability belongs to nobody.** Anything written only into the in-memory index since the last load is lost when that process exits. 5. **There is no single answer to "what is in the index?"**, which makes every later bug report about a missing or stale result hard to reproduce. ## Workarounds that keep it in-process, and why they disappoint | Workaround | What it fixes | What it leaves broken | |---|---|---| | Rebuild from the source of truth on every start | Restarts and new replicas converge eventually | Startup time grows with the corpus; writes between rebuilds still diverge | | Ship a prebuilt index file with each release | Identical, fast-loading copies on every replica | No live writes at all between releases | | Broadcast each write to every replica | Replicas usually agree | A replica that is down misses writes; ordering differs; a new replica needs a full copy from somewhere | | Sticky routing per user | One user sees their own writes | Other users still miss them; a replica restart loses the user's view | The broadcast option deserves a warning: once it needs a durable write log, ordering and a catch-up path for late replicas, the team is building a replicated vector server without the engineering that goes into one. ## The way out The fix is to give the index **one owner that every replica talks to**. The candidate shapes are an embedded store whose files a single writer process owns, a vector server the team runs, a managed vector service, or a vector extension inside the database already in production. Each moves durability, concurrency and upgrades to a different owner, and that choice is a separate question from this one. The one exception is the prebuilt read-only artifact, which stays in-process on purpose and accepts that live writes wait for the next release. ## How to answer in an interview Lead with the conditions that make the in-process shape correct, then name the first failure at several replicas (divergent writes), and finish with the move: either freeze the index into a per-release artifact or move it to a shared owner. Strong answers name divergent writes as the first failure and observe that the memory bill scales with the replica count.

  • Could you broadcast every write to all replicas to keep their in-memory indexes in sync?
    You can, but it is a fragile replication scheme. A replica that is restarting misses writes, two writes can arrive in different orders on different replicas, and a new replica needs a complete copy from somewhere. Once you add a durable write log and a catch-up path to fix that, you are running your own vector server without the testing behind a real one.
  • When is shipping a prebuilt index file with each deploy the right answer?
    When the corpus changes on a release cadence and live writes are not needed. A build job creates the index from the kept vectors, the file is versioned with the release, and each replica loads it at startup, so every copy is identical. The costs are startup time that grows with index size and staleness until the next release.

Each replica is a clerk with a private notebook copied from the master list at the start of the shift. A note one clerk adds never reaches the others' notebooks, and a clerk who arrives mid-shift starts with a fresh copy that knows nothing of the day's additions.

  • Four replicas of the same code share one in-memory index automatically.
  • Sticky sessions fix divergence between replicas' in-memory indexes.
  • Anything added to an in-memory index survives a process restart.
  • In-process indexes are too slow for production, so they are only for prototypes.
  • Adding replicas adds search capacity without adding index memory.
open as a page

A vector store at its memory limit or its query-rate limit: when do you add shards, when replicas, and what does a replica cost?

level: juniorimportance: must knowfreq 55%

basics

~20 s

Add shards when the index no longer fits: each holds a slice of the vectors. Add replicas when queries outrun one copy: each replica is a full copy of a shard's vectors and index, multiplying memory and repeating every write.

open as a page

Why can an acknowledged upsert be missing from a vector store's search results, yet be visible at once in a transactional database?

level: juniorimportance: must knowfreq 50%

basics

~20 s

Many vector stores acknowledge a write once it is accepted or logged, then apply it to searchable segments and replicas asynchronously. A database that updates its vector index inside the transaction makes the row searchable at commit, at least on the primary.

open as a page

Why does 'we can always rebuild the vector index from source documents' mean a recovery time of hours, and what would you keep to skip re-embedding?

level: middleimportance: must knowfreq 42%

basics

~20 s

Rebuilding from source re-reads, re-parses and re-embeds every chunk before the index is built, and embedding millions of chunks under a throughput limit takes hours. Storing each chunk's vector, text and model version durably cuts a rebuild to loading plus index build.

open as a page

After restoring a vector store from a week-old snapshot, which deletions and permission revocations come back, and how do you re-apply them before serving?

level: middleimportance: must knowfreq 38%

basics

~20 s

Every chunk deleted and every permission revoked after the snapshot comes back, along with stale versions of updated documents. Keep an ordered journal of changes outside the store, replay it from the snapshot time with traffic closed, verify, then serve.

open as a page

Why does an embedded vector store usually allow only one writing process, and how do you work within that without changing products?

level: middleimportance: must knowfreq 50%

basics

~20 s

An embedded vector store is a library writing local files with no server to order writers, so most lock out a second writer or assume one. Funnel all writes through one owning process, or run the same engine as a server.

open as a page

Self-run vector database server or managed vector service: who owns durability, upgrades and capacity in each, and what do you give up?

level: middleimportance: must knowfreq 55%

basics

~20 s

Self-running a vector server leaves disks, backups, upgrades, memory sizing and on-call with your team. A managed service takes machines, replication, patching and scaling; you give up upgrade timing, tuning and placement, and your own mistakes stay yours.

open as a page

Why does a vector store with randomly placed shards send every query to every shard, and which placements let a query skip shards?

level: middleimportance: must knowfreq 50%

basics

~20 s

Random or hashed placement can put any vector on any shard, so a query's neighbours could be anywhere and every shard is searched. Placing by tenant key skips shards exactly; placing by vector cluster skips them approximately, risking recall.

open as a page

A user deletes a confidential document, but its chunk still appears in assistant answers minutes later: what can cause it, and how do you tell which?

level: seniorimportance: must knowfreq 45%

basics

~20 s

The delete may still be applying, a lagging replica may still serve the chunk, the delete may have matched no stored record, or the text may come from a cache or chat history. Look up the cited chunk's ID on each replica to tell which.

open as a page

Why can a vector store usually move an index shard to another node by copying it, but not split one without rebuilding its index?

level: middleimportance: should knowfreq 40%

basics

~20 s

A moved shard keeps every vector, so its index stays valid and copies as files. A split strands graph links across the cut, so each half needs a new index, though segment-based stores can sometimes reassign whole segments instead.

open as a page

A deleted vector no longer appears in search results but stays on disk until compaction: what does that mean for a data erasure request?

level: middleimportance: should knowfreq 35%

basics

~20 s

Hidden is not erased. A tombstone stops searches returning the record, but its vector and stored text stay in segment files, and often in older versions and logs, until compaction and cleanup run; erasure is complete only after that.

open as a page

What do a vector store's wait-for-apply write flag and per-request read consistency level buy, and what do they cost?

level: middleimportance: should knowfreq 40%

basics

~20 s

A wait-for-apply flag holds the write's acknowledgement until the write is searchable, buying read-after-write for writer latency and bulk throughput. A stronger read consistency level makes a query consult more replicas or wait for them, buying freshness for latency and availability.

open as a page

A vector-store snapshot is intact, but the hosted embedding model that produced its vectors has been retired; what is that snapshot still worth?

level: seniorimportance: should knowfreq 25%

basics

~20 s

New text queries can no longer be embedded into the snapshot's vector space, so query search over it is lost. Its chunk text and metadata still seed a re-embed with a new model, and item-to-item similarity among stored vectors still works.

open as a page

An overnight ingest job wrote corrupt vectors into 2% of a vector index; how do you undo it without losing the day's good writes?

level: seniorimportance: should knowfreq 34%

basics

~20 s

Restoring last night's snapshot would discard the day's good writes and revive deletes, so repair surgically: find the bad job's records by their load ID, hide them, then rewrite only those IDs with correct vectors from kept copies or a fresh embed.

open as a page

How do you move a live vector index from an embedded store to a server or managed service without re-embedding the corpus?

level: seniorimportance: should knowfreq 40%

basics

~20 s

Export each record's ID, full-precision vector and metadata with the model identity, metric and normalisation, bulk-load them so the target builds its own index, verify counts, IDs and sample-query overlap, then catch up later writes and switch reads.

open as a page

A team's dedicated managed vector cluster averages five percent utilisation; when does a serverless vector index cost less, and what limits come with it?

level: seniorimportance: should knowfreq 38%

basics

~20 s

A dedicated cluster bills provisioned nodes whether busy or idle; a serverless index bills storage plus each read and write. Serverless wins while monthly request charges stay below the cluster's fixed bill, but brings rate quotas, caps and fewer tuning knobs.

open as a page

A vector store places vectors on shards by nearest cluster centroid and routes each query to the closest few shards: what recall does that risk, and how do you bound it?

level: seniorimportance: should knowfreq 30%

basics

~20 s

Neighbours near a cluster boundary, and queries far from every centroid, land on shards the router skipped, so recall drops without any error. Bound it by measuring against full fan-out, probing more shards, duplicating boundary vectors and re-clustering when data drifts.

open as a page

Adding a replica of a large graph-indexed vector shard: do you rebuild the index on the new node or ship the built index files?

level: seniorimportance: should knowfreq 30%

basics

~20 s

Shipping the built files is usually faster and gives an identical replica, but needs a consistent copy, a compatible index format and a replay of writes since the copy. Rebuilding often costs hours of CPU and yields a slightly different graph.

open as a page

How does an ingestion pipeline prove a delete landed on every replica of a vector store, rather than merely that it was acknowledged?

level: seniorimportance: should knowfreq 32%

basics

~20 s

Read it back: after the acknowledgement, check each replica with exact ID lookups and a document-filter count, poll until every one reports zero, and record when. An approximate search that no longer returns the chunk proves little on its own.

open as a page

A user uploads a file to a retrieval-augmented assistant on an eventually visible vector store and asks about it at once: how do you give them read-after-write?

level: seniorimportance: should knowfreq 38%

basics

~20 s

Scope the guarantee to that user: wait for apply on the upload's final writes before marking the file ready, carry a write position into their session's reads, or overlay their fresh chunks with an exact search, merged and deduplicated by chunk ID.

open as a page

As the lead, would you make snapshot restore or rebuild from kept sources your team's standing recovery bet for a vector store, and what decides it?

level: principalimportance: should knowfreq 30%

basics

~20 s

Bet on the path your drills prove meets the recovery time: snapshots usually restore fastest but revive deletes and stay tied to their encoder; rebuilds from sources are current but slow. Many teams keep both, plus stored vectors between.

open as a page

Should a team keep embeddings in the relational database it already runs, through a vector extension, or adopt a dedicated vector store?

level: principalimportance: should knowfreq 48%

basics

~20 s

Start in the relational database already in production while vectors fit beside its transactional load: one system, one transaction, existing backups. Move to a dedicated vector store when measured contention, scale or missing features demand it, accepting a sync path.

open as a page

A search feature has ten thousand tenants on one vector store near its memory limit: an index per tenant or pooled shards by tenant key, and what happens when one tenant outgrows its pool?

level: principalimportance: should knowfreq 40%

basics

~20 s

Neither pure shape usually holds: ten thousand tiny indexes each pay fixed overhead, and pooled shards let one tenant crowd its neighbours. Most teams pool small tenants by tenant key and promote large ones to dedicated shards past a threshold.

open as a page

A sharded vector store snapshots each node on its own and keeps chunk text in a separate store; why can the restored system be inconsistent, and how do you prevent it?

level: seniorimportance: nice to knowfreq 22%

basics

~20 s

Copies snapshotted minutes apart freeze each shard and the text store at different moments, leaving half-present documents, mixed versions and vectors with no text. Take one cut with writes paused, or record each copy's log position and replay all to one point.

open as a page