In an Elasticsearch knn search, how do k and num_candidates differ?

## The two numbers do different jobs A `knn` search in Elasticsearch carries two integers that beginners routinely conflate. `k` is a result-set size: how many nearest neighbours you want. `num_candidates` is a search-effort dial: how hard each shard works to find them. Confusing them produces either a slow search that returns ten documents at the cost of an exhaustive scan, or a fast search that quietly misses the best answers. ## How approximate search actually runs Elasticsearch stores vectors in an HNSW graph — a layered proximity graph where each vector is linked to its near neighbours. Searching means entering the graph at an upper layer, greedily walking toward the query vector, and descending. The walk keeps a bounded priority queue of the best candidates seen so far; `num_candidates` is that bound. When the queue stops improving, the walk stops and the top `k` entries are returned. Because the walk is greedy over a bounded queue, it is *approximate*: a true nearest neighbour sitting in a poorly connected part of the graph can be missed. A larger queue means the walk keeps more plausible directions alive and backtracks more, so it finds more of the true top-k. In the limit, a queue as large as the segment is exhaustive search. ## The per-shard multiplier This is the detail interviewers probe. The graph walk happens **per shard** (in fact per segment, with results combined within the shard). Each shard explores up to `num_candidates` vectors and returns its own top `k`. The coordinating node then merges those lists and keeps the global top `k`. So: - Total work scales with `num_candidates × number of shards`, not with `k`. - Returned hits are `k`, not `k × shards`. - A heavily over-sharded index makes vector search expensive without making it better, because each small shard pays the full traversal cost. A useful corollary: adding shards tends to *raise* recall slightly, since each shard independently searches its own smaller graph, but it raises cost faster. Vector-heavy indices generally want fewer, larger shards than the reflexes learned from log indices suggest. ## Constraints and defaults `num_candidates` must be greater than or equal to `k`; asking for 100 neighbours out of a 50-candidate queue is rejected. It is also capped at 10,000 per shard, which is the ceiling on how much recall you can buy by turning this one dial. In recent versions `k` may be omitted, in which case it follows the request's `size`. Do not rely on a default for `num_candidates` — set it explicitly so that recall is a decision rather than an accident. ## Tuning method The honest way to pick `num_candidates` is measurement, not folklore: 1. Build a query set that resembles production traffic. 2. Compute exact ground truth once — brute-force the same vectors with `script_score` and vector functions, or an offline exact search. 3. For each `num_candidates` value, compute recall@k: the fraction of the true top-k that the approximate search returned. 4. Plot recall against p99 latency. Recall climbs steeply and then flattens; the elbow is your setting. If recall is still poor at the ceiling, `num_candidates` is the wrong dial. Raise `m` in `index_options` so the graph is better connected, raise `ef_construction` so it is better built, or reconsider quantization if a lossy vector representation is blurring near neighbours. ## Interactions worth knowing With a `filter` on the kNN search, candidates that fail the filter do not count toward your top-k, so filtered searches typically need a larger `num_candidates` to reach the same recall. When kNN is one leg of a hybrid retrieval, the kNN leg's `k` sets how many documents that leg can contribute to fusion — too small a `k` and a good lexical match never gets a vector-side rank to fuse with. And when kNN is expressed as a `knn` query inside the query DSL rather than as the top-level search option, the number of returned documents follows the surrounding request's `size` rather than a `k` you set on the clause, which is a common source of confusion when porting a request between the two forms. ## The short version for an interview `k` is what you ask for; `num_candidates` is what you pay for it; the payment is per shard; and the only defensible way to set it is to measure recall against exact search.