skip to content

A user uploads a file to a retrieval-augmented assistant on an eventually visible vector store and asks about it at once: how do you give them read-after-write?

level: seniorimportance: should knowfreq 38%

answer

  1. scope it to one user
  2. wait, then mark ready
  3. carry the write position
  4. exact overlay of fresh chunks
  5. deduplicate by chunk ID

basics

~20 s

Scope the guarantee to that user: wait for apply on the upload's final writes before marking the file ready, carry a write position into their session's reads, or overlay their fresh chunks with an exact search, merged and deduplicated by chunk ID.

solid answer

~50 s

The guarantee is needed by one user for one file, so I would not slow the whole store for it. The default is **wait-then-ready**: send the upload's last writes with the store's wait-for-apply option at a scope covering the replicas queries reach, and show the file as processing until it returns. If the store supports it, the session can instead carry the write's **position** so that user's reads wait for it. Where users ask within seconds of a large upload, I would add an **overlay**: keep the new chunks in a per-session buffer, score them exactly against each query, merge with the store's hits under the same access filter, deduplicate by chunk ID, and drop them once the store serves them. Sticky routing to the writing node is weak, because that node may not have applied the write either.

code

pseudocode · 10 lines
pseudocode
function search_for_user(user, query_vec, k):
    store_hits  = store.search(query_vec, k, filter: access_allows(user))
    fresh       = recent_uploads(user)                     # not yet confirmed served by the store
    buffer_hits = exact_top_k(fresh, query_vec, k, metric: store.metric)
    buffer_hits = [h for h in buffer_hits if access_allows(user).matches(h)]   # same check as the store's
    best = {}
    for hit in store_hits + buffer_hits:
        if hit.chunk_id not in best or hit.score > best[hit.chunk_id].score:
            best[hit.chunk_id] = hit                       # one copy per chunk ID
    return top_k_by_score(values(best), k)

go deeper

for a junior

Recall that a freshly uploaded file may not be searchable yet on many vector stores, and that the product can wait before marking it ready.

for a middle

Explain why sticky routing is weak on a store that applies its index asynchronously, and what a session or write-position read adds.

for a senior

Design the overlay: exact scoring of fresh chunks, the same access filter, deduplication by chunk ID, score scale, expiry and deletes.

for a principal

Pick the default for the product, usually wait-then-ready, and reserve overlays or stronger reads for flows where measured user behaviour needs them.

## One user's problem, not the cluster's A user uploads a file to a retrieval-augmented assistant and immediately asks a question about it. The ingestion pipeline has chunked and embedded the file and the vector store has acknowledged the writes, but on a store that applies writes asynchronously those chunks may not be searchable for seconds or minutes. The user gets an answer that ignores their file, which reads as a broken product. The guarantee missing is **read-after-write** (read-your-writes) for one user. The general theory of session guarantees belongs with consistency models; the vector-store question is how to give this one user that guarantee without slowing every write and every query in the system. ## Ways to close the gap 1. **Wait, then say ready.** Send the upload's final writes with the store's wait-for-apply option, at a scope that covers the replicas queries will reach, and show the file as "processing" until it returns. Simple and correct; the cost is the user's wait, paid once per upload. 2. **Carry the write position into the session.** Some stores return a position or timestamp for a write and accept a read option that waits until the serving replica has applied up to it, or offer a session level that includes the client's own writes. The user's later queries carry that marker; everyone else reads cheaply. 3. **Route the user's reads to the node that took the write.** Weak on vector stores: even that node may not have applied the write to its searchable segments yet, and it fails when the node does. 4. **Overlay the just-uploaded chunks.** Keep the upload's chunks and vectors in a small per-session buffer, score them **exactly** against each query, merge with the store's hits, and drop each chunk from the buffer once the store is confirmed to serve it. 5. **Skip retrieval for small files.** If the whole file fits the model's context window, put it in directly for the next few turns; retrieval only has to catch up later. | Approach | Freshness | Cost | Weak point | |---|---|---|---| | Wait, then mark ready | exact | the uploader's wait | slow on large files | | Session or position read | exact, if the store supports it | slower reads for that user | store-specific | | Sticky routing | partial | little | node may not have applied it | | Exact overlay | exact | buffer and merge code | merge must be right | | File in the context | exact | prompt tokens | only small files | ## Why the overlay is cheap Take a scenario: a file splits into 300 chunks embedded as 1,024-dimension 32-bit vectors. The buffer holds 300 x 1,024 x 4 bytes = 1,228,800 bytes, about 1.2 MB. Scoring a query against all of them is 300 x 1,024 = 307,200 multiply-adds, well under a millisecond on one core. Exact brute force, which gets expensive over millions of vectors at high query rates, is trivial over a few hundred. ## Merging the overlay correctly - **Deduplicate by chunk ID.** Once the store catches up, the same chunk arrives from both lists; keep one copy. - **Score on the same scale.** Use the store's similarity metric. If the store returns scores from compressed vectors, rescore both lists exactly before merging, or the overlay's exact scores will sit on a different scale. - **Apply the same access filter.** The buffer must pass the same permission check as store hits, or it becomes a path around access rules. - **Expire and delete.** Remove a chunk from the buffer once a lookup on the serving replicas returns it, and at once if the user deletes the file. The pseudocode shows the merge: store hits and exact buffer hits pass the same access check, deduplicate by chunk ID keeping the higher score, then take the top k. ## Choosing - For most products, **wait-then-ready** on uploads is the default: one wait per upload, visible progress, no new read path. - Add the **overlay** where users routinely ask within seconds and files are large enough that waiting hurts. - Use **session or position reads** when the store offers them and the team wants no extra code path. - Avoid making every write wait or every read strong to solve one user's problem; that moves the cost onto bulk loads and every other query.

  • Why not make every write in the system wait for apply instead?
    Because only this user's next question needs the guarantee. A global wait slows every bulk load and background sync for no reader's benefit and deepens the apply backlog. Wait on the upload's final writes, or carry its write position into this user's session, and leave the bulk path asynchronous.
  • What check must the overlay never skip?
    The access filter the store's hits pass. Every buffered chunk must pass the same permission check before it is merged, or the overlay becomes a path around access rules. Buffered chunks must also be removed at once if the user deletes the file, or the overlay serves a deleted document.

  • Make every write in the cluster wait for apply so uploads are always fresh.
  • Pinning the user to the node that took the write guarantees they see it.
  • Overlay hits can be appended after the store's hits without deduplication.
  • Nothing can be done; tell the user to wait a minute and ask again.
  • Read-after-write requires moving the whole corpus to a transactional database.