skip to content

What do a vector store's wait-for-apply write flag and per-request read consistency level buy, and what do they cost?

level: middleimportance: should knowfreq 40%

answer

  1. move the lag, don't remove it
  2. writer waits or reader waits
  3. receiving node versus every replica
  4. slowest replica sets the latency
  5. scope it to the paths that need it

basics

~20 s

A wait-for-apply flag holds the write's acknowledgement until the write is searchable, buying read-after-write for writer latency and bulk throughput. A stronger read consistency level makes a query consult more replicas or wait for them, buying freshness for latency and availability.

solid answer

~50 s

Both move the visibility window into someone's latency budget. With a **wait flag** the writer blocks until the store has applied the write, on the receiving node or, depending on the store, on its replicas, so a reader that follows can see it; the cost is per-call latency, and waiting on every small batch of a bulk load wrecks throughput. A **per-request read consistency level** lets a reader ask for more: consult a majority or all replicas, wait until its replica has applied up to a point, or include its own session's writes. That cost lands on reads: latency set by the slowest replica, failed queries when a required replica is down, and more work per query. Use both on the few paths that need read-after-write, such as an upload or a compliance delete, not as the cluster default.

code

json · 17 lines
json
[
  {
    "op": "upsert",
    "collection": "support-chunks",
    "records": [
      {"id": "doc-118#3", "vector": [0.12, -0.04, 0.33], "payload": {"doc_id": "doc-118"}}
    ],
    "wait_until": "applied_on_replicas"
  },
  {
    "op": "search",
    "collection": "support-chunks",
    "vector": [0.10, -0.02, 0.31],
    "top_k": 5,
    "read_consistency": "majority"
  }
]

go deeper

for a junior

Recall the two knobs: the writer can wait until a write is searchable, or the reader can ask for a fresher answer.

for a middle

Explain what the flag waits for on the store you know, receiving node or replicas, and why every read level above one replica costs tail latency.

for a senior

Work out the bulk-load cost of waiting per batch, scope the guarantee to the paths that need it, and say where ordering lets you wait once.

for a principal

Decide the team's defaults: cheap writes and reads cluster-wide, with named exceptions such as uploads and compliance deletes, and the measurements that justify them.

## Two knobs, two sides of the same window When a vector store applies writes asynchronously, someone has to absorb the gap between "acknowledged" and "searchable". Stores commonly offer two places to put it: - a **wait-for-apply flag** on the write: the call does not return until the store has applied the write to something searches read; - a **per-request read consistency level** on the query: the reader asks for a fresher answer than the cheapest one. Neither removes the lag. Each moves it from "a reader might see stale results" into someone's latency budget, and each has a different scope. ## What a wait flag actually waits for The flag's meaning varies by store, so read the documentation for the exact call. Common shapes: 1. **Received** - the cheapest acknowledgement, with no wait at all. 2. **Logged** - the write is durable in the write-ahead log but may not be searchable. 3. **Applied on the receiving node** - searches served by that node can return it. 4. **Applied on the replicas the write was copied to** - searches served by any of those can return it. The gap between 3 and 4 is the usual trap: a flag that covers only the node that took the write still leaves a lagging replica serving stale results. Some stores pair the wait with a **write consistency** setting that decides how many replicas must accept the write before it is acknowledged; together they set how far the guarantee reaches. ## What read consistency levels do A read level sets how many replicas a query consults, or how fresh the replica it hits must be: - **One replica** - fastest, may be stale. - **A majority or all replicas** - the query consults several and takes the freshest answer; the general quorum theory behind this belongs with replication, not here. - **Bounded staleness** - the query waits until the replica it hits has applied writes up to a time or position, or accepts data no older than a bound. - **Session** - the store makes a client's later reads include that client's own writes, usually by tracking a write position for the session. Stores differ in which of these they expose and in their defaults; none of the names above is universal. ## The costs side by side | Option | Who pays | What it costs | Where it fits | |---|---|---|---| | Wait flag on every write | writers | per-call latency, throughput on bulk loads | rarely; small interactive writes | | Wait flag on chosen writes | those writers | latency on those calls only | an upload, a compliance delete | | Strong read level everywhere | every reader | tail latency of the slowest replica, failed reads when a required replica is down, more replica work | rarely | | Strong or session read on chosen requests | those readers | the same, on those requests only | reading back one's own write | The bulk-load case is worth computing. A scenario: ten million chunks in batches of 100 is 100,000 calls. If each waited call adds an illustrative 200 milliseconds of apply wait, a single serial loader spends 100,000 x 0.2 s = 20,000 s, about 5.6 hours, idle. Parallel loaders shrink the wall clock but not the waste, and they deepen the apply backlog that every other writer then waits behind. ## Using them well - **Scope the guarantee.** Wait or raise consistency only where a reader needs read-after-write: a user reading back an upload, a delete that must be gone before a confirmation is shown. - **Wait once, not per batch,** where the store applies a shard's writes in order: waiting on the last write to each shard then implies the earlier ones are applied. Check the store documents that ordering; across shards there usually is none. - **Keep the cluster default cheap.** A strongest-level default makes every query as slow as the slowest replica and lets one unhealthy replica fail reads. - **Keep visibility and durability apart.** Durability comes from the log and write consistency; the wait flag is mainly about visibility, though on some stores a no-wait acknowledgement returns before the write is even logged, so check both meanings. - **Measure both sides.** Track write latency with and without the flag and read tail latency per level, so the choice is made on numbers, not habit.

  • An upload writes 40 batches to one shard. Do you need to wait on every batch?
    Often not. If the store applies each shard's writes in order, waiting on the last batch implies the earlier 39 are applied too, so you pay the wait once. Confirm that ordering guarantee in the store's documentation; writes spread across shards usually have no order between them, so wait on the last write to each shard.
  • Why not make the strongest read consistency the cluster default?
    Because every query then waits for the slowest replica and fails while any required replica is down, for a guarantee most reads do not need. Keep the default cheap and raise the level per request on the paths that need fresh results, such as a user reading back their own upload.

  • A wait flag guarantees every replica has applied the write, on every store.
  • Setting the strongest read level as the default costs nothing on a healthy cluster.
  • Waiting on every small batch of a bulk load costs nothing worth measuring.
  • One strong read makes every other client's reads fresh as well.
  • Reading from all replicas improves recall by merging their different neighbours.