How does Qdrant's scroll API differ from search when exporting all points?
answer
- no query vector involved
- ordered walk, not ranking
- returns a page plus an offset
- loop until the offset is None
- vectors are excluded by default
basics
~20 sScroll enumerates stored points in id order with no query vector and no similarity scoring, returning a page plus an offset for the next call. Search ranks by distance to a query vector and is bounded by a limit, so it can never enumerate a collection.
solid answer
~50 s`client.scroll(collection_name="docs", limit=1000, with_payload=True, with_vectors=False)` returns a tuple: the page of points, and a `next_page_offset`. You loop, passing that offset back in as `offset=`, until it comes back `None`. There is no query vector involved — points come out ordered by id, so the traversal is complete and stable rather than ranked. That is the difference that matters: similarity search returns the `limit` best matches for a vector and has no notion of "the rest", so raising `top_k` to a huge number to dump a collection is both wrong and expensive. Scroll accepts `scroll_filter` to restrict which points are visited, and an `order_by` on a payload field if you need a different order — that requires an index on the field. Keep `with_vectors=False` unless you actually need the embeddings, since vectors dominate response size; the common uses are export, backfill, re-embedding and auditing.
code
python · 19 linesfrom qdrant_client import QdrantClient
client = QdrantClient(url="http://localhost:6333")
offset = None
total = 0
while True:
points, offset = client.scroll(
collection_name="docs",
limit=1000,
with_payload=["doc_id", "lang"],
with_vectors=False,
offset=offset,
)
total += len(points)
if offset is None:
break
print(total, client.count("docs", exact=True).count)go deeper
Know that scroll pages through stored points without a query vector, and that it returns both the points and an offset to continue with.
Explain the loop-until-offset-is-None pattern, the role of limit, with_payload and with_vectors, and why search with a giant limit is not an enumeration tool.
Show you use scroll for real jobs — re-embedding, backfill, reconciliation — with checkpointed offsets for resumability, and that you know it is a live walk rather than a consistent snapshot.
Own the choice between a live scroll and a snapshot for bulk data movement, weighing consistency guarantees, load on a serving cluster, and how much of the corpus you can afford to reprocess after a failure.
## Two different jobs Similarity search answers "which points are most like this vector?" — it takes a query vector, walks the index, and returns the top matches with scores. It is inherently bounded and inherently ranked. There is no meaningful continuation of it: asking for results 10,000 to 11,000 of an approximate nearest-neighbour search is neither cheap nor stable. Scroll answers a different question: "give me the points, all of them, in a defined order." No query vector, no scoring, no index traversal — just an ordered walk over stored points. Reaching for search to enumerate a collection is a category error, and it is one interviewers probe deliberately. ## The call and its cursor `client.scroll(...)` returns a **tuple** of `(points, next_page_offset)`. The first element is the page; the second is the value you feed back as `offset=` on the following call. When `next_page_offset` is `None`, you have reached the end. ``` offset = None while True: points, offset = client.scroll("docs", limit=1000, offset=offset) handle(points) if offset is None: break ``` The offset is a point id, not an opaque server-side session — nothing expires, and nothing has to be released. That means a scroll can be checkpointed: persist the last offset, and a crashed export resumes from there rather than starting over. It also means the paging is keyset-style rather than numeric-skip, so page N does not get more expensive than page 1. Because the walk is ordered by id and the offset is an id, concurrent writes behave predictably: a point inserted with an id before your current position will not be visited, and one inserted after it will. A scroll is not a transactional snapshot of the collection, and you should say so when asked — for a consistent point-in-time copy you take a snapshot instead. ## The parameters that matter `limit` is the page size. Bigger pages mean fewer round trips and larger responses; a few hundred to a few thousand is typical, tuned by payload size. `with_payload` and `with_vectors` control what comes back. Payload is included by default, vectors are not — and that default is right. Vectors are the bulk of the bytes, and most scroll use cases (auditing metadata, backfilling a field, exporting ids) do not need them. Turn `with_vectors=True` on only for genuine vector export, and drop the page size when you do. `with_payload` also accepts a list of field names, so you can fetch just the two keys you need instead of a fat payload. `scroll_filter` restricts the walk to points matching a condition, which is how you enumerate a subset — one tenant's points, or everything ingested before a date — without pulling the whole collection and discarding most of it client-side. `order_by` changes the traversal order to a payload field rather than id. It requires that field to have an index, and it is what you use when the export must be chronological rather than id-ordered. ## Related read calls Scroll is one of three ways to read without searching. `client.retrieve(collection_name, ids=[...])` fetches specific points by id — the right call when you already know what you want, and much cheaper than scrolling to find them. `client.count(collection_name, exact=True)` returns the point count; use it to know how big a scroll will be, or to verify one finished. And a collection snapshot is the right tool when you need a consistent copy rather than a live walk. ## Where scroll shows up in real work **Re-embedding.** Scroll the source collection with payload, run the new model over the stored text, and upsert into a new collection. Because scroll is resumable by offset, a job that dies at 60% picks up where it stopped. **Backfill.** A new payload field needs computing for existing points: scroll ids and the inputs, compute, then `set_payload` in batches. **Audit and reconciliation.** Compare the ids present in Qdrant against your primary datastore to find drift — points whose source rows were deleted, or rows never ingested. **Export and migration.** Dump points to a file for transfer to another environment when a snapshot is not appropriate — for instance when the target has a different collection configuration. ## Anti-patterns The headline one is `query_points(..., limit=1000000)` to "get everything". It forces an enormous approximate search, returns results in similarity order relative to an arbitrary vector, and gives no guarantee of completeness. Second is scrolling with `with_vectors=True` when only payload is needed, which multiplies transfer volume for nothing. Third is discarding `next_page_offset` and trying to page with an incrementing counter — the offset is an id, and an integer skip is not what this API takes.
- Is a scroll a consistent snapshot of the collection?No. It is a live ordered walk, so writes that happen while it runs may or may not be seen depending on where their ids fall relative to your current offset. If you need a point-in-time consistent copy — for backup or migration — take a collection snapshot instead and read from that.
- How do you resume an export that crashed halfway through?Persist the last `next_page_offset` you successfully processed and pass it as `offset` on restart. The offset is a point id rather than a server-side session, so nothing has expired and no state was held on the server while you were gone.
- Why is with_vectors left off by default?Because vectors dominate response size — a thousand 1536-dimension float vectors is several megabytes per page, while the payload fields you usually want are a fraction of that. Most scroll workloads audit or backfill metadata and never need the embeddings, so paying for them by default would be the wrong trade.
- Uses search with a huge limit to dump an entire collection
- Ignores next_page_offset and pages with an incrementing integer
- Assumes scroll gives a transactionally consistent snapshot
- Requests vectors on every page when only payload is needed
- Thinks scroll ranks results by similarity
qdrant-client 1.19
context
- Q-08115What are the parts of a Qdrant point, and which id types does it accept?level: juniorimportance: must knowfreq 80%