Why must $search be the first stage of an Atlas aggregation pipeline?

## Two processes behind one query Atlas Search and Atlas Vector Search are not part of the `mongod` query engine. An Atlas cluster carrying search indexes also runs **`mongot`**, a Lucene-based process that lives either alongside `mongod` on the same node or on separate **Search Nodes**. `mongot` maintains its own inverted index (for `$search`) or vector index (for `$vectorSearch`), kept current by following the collection's change stream. The documents themselves stay in `mongod`'s storage engine. When a pipeline begins with `$search`, `mongod` ships the search specification to `mongot`. `mongot` evaluates it against the Lucene index and returns a stream of document identifiers, each with a relevance score. `mongod` then fetches those documents by `_id` and passes them into the rest of the pipeline. ## Why that forces first position An ordinary aggregation stage transforms a stream of documents it is given. `$search` does the opposite: it *creates* the stream, out of an index that another process owns. There is no protocol for handing mongot a set of already-selected documents to search within — the contract is "here is a query, return ids and scores". So the stage has to sit at the source of the pipeline, and the server enforces it: a `$match` (or anything else) placed ahead of `$search` produces an error, not a slower plan. The restriction is about *a* pipeline, not *the* pipeline. `$search` may legally be the first stage of a `$unionWith` or `$lookup` sub-pipeline, because each of those opens a fresh pipeline over a collection. That is exactly what makes hybrid search patterns expressible. It may not appear inside `$facet`. ## The mistake this causes The common bug is filtering after the fact: ```javascript { $search: { text: { query: "laptop", path: "title" } } }, { $match: { inStock: true } }, { $limit: 20 } ``` mongot chose its best matches with no knowledge of `inStock`, so the `$match` can throw most of them away and you end up with a handful of results — or none — even though plenty of in-stock laptops exist. The fix is to push the predicate into the search itself, as a `filter` clause of the `compound` operator (for `$vectorSearch`, the stage's own `filter` option). A `filter` clause is evaluated by mongot while it selects, and unlike `must` or `should` it contributes nothing to the score. ## Reading the search metadata Because the score is produced by mongot rather than stored on the document, later stages read it through `$meta`: `{ $meta: "searchScore" }` for `$search`, `{ $meta: "vectorSearchScore" }` for `$vectorSearch`, and `{ $meta: "searchHighlights" }` when the query carries a `highlight` option. Setting `scoreDetails: true` on the search and projecting `{ $meta: "searchScoreDetails" }` explains how a score was assembled, which is the fastest way to debug ranking. Facet and count metadata surface separately, through the `$searchMeta` stage or the `$$SEARCH_META` variable. ## Skipping the document fetch The id-then-fetch round trip costs a lookup in `mongod` for every hit. If the fields you display are small, mark them `storedSource` in the index definition and pass `returnStoredSource: true` in the query; mongot then returns the stored copy directly and `mongod` skips the fetch. The trade is index size and the fact that stored copies lag the collection exactly as the index does. ## What to say in an interview Name mongot, describe the ids-and-scores hand-off, and connect it to the two things that actually bite in production: predicates must live inside the search rather than in a following `$match`, and search results are eventually consistent with the collection because the index is fed by a change stream.