In hybrid search, why does a selective filter hurt the vector leg more than the keyword leg?
answer
- one leg intersects, the other approximates
- selectivity cuts cost on one side only
- the graph was built over everything
- most visited neighbours get discarded
- fused results quietly become lexical-only
basics
~20 sA keyword engine treats a filter as one more postings list to intersect, so it gets cheaper and stays exact as selectivity rises. An approximate vector index must discard matches after searching, or traverse a structure built over every vector, so recall falls instead.
solid answer
~50 sThe two legs handle restriction with opposite cost curves. Lexically, a filter is just another posting list intersected with the query's — skip lists make the intersection cheaper the more selective the filter is, and the result is exact. Approximate vector search has no such mechanism. Post-filtering searches the whole space for the nearest k and then throws away non-matching hits, so a filter matching a tiny fraction of documents returns almost nothing. Filtered traversal walks a structure built over *all* vectors while consulting an allow-list, so most visited neighbours are discarded and the effective search must widen sharply, or recall collapses; the matching subset can even be poorly connected in the graph. Exact pre-filtering is correct but only affordable when the matched set is small. The hybrid consequence is what interviewers are after: on filtered queries the lexical leg comes back full and exact while the vector leg comes back thin, so the fused list quietly becomes lexical-only — and unfiltered offline evaluation never shows it.
go deeper
Know that restricting a search to a subset of documents is cheap for keyword search and awkward for vector search, and that filtering after a vector search can leave you with far fewer results than you asked for.
Explain the three filtering strategies — post-filter, exact pre-filter scan, and filtered traversal — and why an approximate index built over all vectors cannot exploit selectivity the way a postings intersection can.
Diagnose the fused-result consequence: on filtered queries the vector leg thins out and the ranking silently becomes lexical. Show how you would measure per-selectivity recall against exact search and switch strategy on estimated cardinality.
Own the data-modelling call — partitioning the vector corpus by tenant, language or region so that common filters become index selection — and the policy that authorization filters are correctness boundaries, never post-hoc result trimming.
## The lexical side: filters are free, and get freer In an inverted index a filter is structurally identical to a query term. `category:books` is a posting list of document ids; the query's terms are posting lists of document ids; retrieval intersects them. Intersection is driven by the *shortest* list, and skip structures let the engine leap over long runs of the other lists. The more selective the filter, the fewer documents survive to be scored, and the faster the query runs. The answer is exact: every matching document is considered, no more and no less. This is why filtering feels like a non-issue to anyone who has only operated keyword search, and why the vector leg's behaviour is such a common production surprise. ## The vector side: three strategies, three problems Approximate nearest-neighbour indexes are built to answer one question — "nearest to this point" — over the *whole* vector set. A filter is foreign to that structure, and every way of bolting it on has a defect. **Post-filter.** Run the ordinary search for the top k, then drop hits that fail the filter. Cheap and simple. Catastrophic under selectivity: if the filter matches 1% of documents and you asked for 10, you should expect roughly a handful of survivors from a candidate pool of 100 — and if the matching documents also happen not to be among the global nearest, you get zero. Systems compensate by over-fetching, which is a guess: you cannot know in advance how far down you must go, and the over-fetch multiplier that works for one filter is wrong for another. **Pre-filter with exact scan.** Resolve the filter first, then compute similarity against every matching vector, brute force. Perfectly exact and perfectly ranked. Its cost is linear in the size of the matched set, so it is excellent when the filter is very selective and unusable when it matches millions. **Filtered traversal.** Search the approximate structure while consulting an allow-list, so only matching candidates enter the result heap. This is what mature engines do, and it is the best option, but it is not free. In a graph-based index the graph was built over all vectors, so the traversal spends most of its work stepping through non-matching nodes just to reach matching ones; the effective exploration factor must grow to keep recall, and cost rises rather than falls with selectivity. Worse, the subgraph induced by the matching documents may be poorly connected — whole regions of matching vectors can be unreachable from the entry point without passing through excluded nodes, so genuinely near neighbours are simply never visited. In a cluster-based index the matching documents may be scattered across many partitions, forcing far more partitions to be probed than the unfiltered query needed. The common thread: **as the filter tightens, lexical retrieval gets cheaper and stays exact, while vector retrieval gets more expensive or less complete.** ## What that does to the fused result Run both legs with the same filter and the imbalance propagates. The lexical leg returns a full, exact, well-ranked list. The vector leg returns fewer candidates, and the ones it returns are further from the true nearest neighbours of the filtered subset. Under rank fusion, the vector leg contributes fewer terms and its candidates are drawn from a degraded ordering; under score fusion, the missing-leg fill values pile up. Either way the fused ranking tilts lexical exactly on the queries where a filter is applied. This failure is invisible in most evaluation programmes because judgment sets are usually built from unfiltered queries, while production traffic is full of filters — a tenant scope, a language, a permission set, a category facet. Semantic quality can be excellent in the lab and mediocre for every logged-in user browsing inside a category. ## Mitigations that actually work **Turn the filter into index selection.** If a field has moderate cardinality and appears in nearly every query — tenant, language, region — partition the vector data by it. The filter then selects which index to search, and inside that index the search is unfiltered and behaves normally. This is the single most effective fix and it is a data-modelling decision, not a tuning one. **Switch strategy on estimated cardinality.** Below some matched-set size, exact scan is both cheaper and better than approximate traversal; above it, filtered traversal wins. Good engines do this automatically from an estimate; if yours does not, route in your own application layer. **Widen the vector leg under filters.** Increase the candidate pool for filtered queries specifically, rather than paying for a large pool on every query. Measure what multiplier restores recall for your selectivity bands rather than picking one. **Measure the vector leg's recall against exact search, with filters applied.** This is the only way to see the problem. Run a sample of filtered production queries through brute-force exact similarity over the filtered subset and compare overlap with what the approximate leg returned. Report it by selectivity band. A leg at 95% recall unfiltered and 40% recall at 0.1% selectivity is a different system, and you will not know which one you shipped without this measurement. **Treat security filters differently.** A permission or tenant filter is a correctness boundary. Post-filtering after the fact is acceptable for a category facet; for authorization it must be enforced before results exist, and the recall cost is simply the price of the boundary.
- When is exact brute-force scan the right way to filter a vector search?When the filter's matched set is small enough that scoring every vector in it is cheaper than an approximate traversal that keeps rejecting candidates — often a few thousand to a few tens of thousands of vectors, depending on dimension and hardware. It is also exactly ranked, which removes the recall question entirely. The right implementation estimates matched cardinality and switches strategy, rather than committing to one.
- How would you measure whether filters are degrading your vector leg?Sample real filtered queries from logs, bucket them by filter selectivity, and for each one compute the true nearest neighbours by brute force over the filtered subset. Compare that ground truth with what the approximate leg returned and report recall per selectivity band. Unfiltered recall numbers tell you nothing about this failure, which is why it survives so long in production.
- Why is partitioning the vector index by tenant better than filtering by a tenant field?Because it converts a filter into index selection. Each tenant's index contains only that tenant's vectors, so the search inside it is unfiltered and its recall and latency behave normally. It also makes the isolation boundary structural rather than a query parameter someone can forget. The costs are per-index overhead and awkwardness at very high tenant counts or very skewed tenant sizes.
The keyword engine has a guest list it can cross-check at the door; the vector index has to walk the whole party looking for people who happen to be on it.
saying these in an interview costs you the question
- Assumes a selective filter speeds up both legs equally
- Thinks post-filtering is equivalent to filtering during the search
- Believes filtering removes approximation error from vector search
- Evaluates vector recall only on unfiltered queries
- Enforces a tenant or permission boundary by post-filtering results