Why should an edge_ngram autocomplete field in Elasticsearch use a different analyzer at search time?

## The shape of the data An `edge_ngram` tokenizer or token filter takes a token and emits every prefix of it between `min_gram` and `max_gram` characters. With `min_gram: 1` and `max_gram: 10`, indexing `"star wars"` (after lowercasing and splitting) stores the terms `s, st, sta, star` and `w, wa, war, wars`. The point is to convert a prefix *search* problem into an exact *term lookup* problem: the user types `sta`, you look up the term `sta`, and you get a posting list — no wildcard scan, no per-query prefix expansion. ## What goes wrong when both sides n-gram If the field declares only `analyzer` and no `search_analyzer`, the same expansion runs on the query. Typing `star` produces four query terms instead of one: - **Precision collapses.** A `match` query ORs its terms, so the term `s` alone pulls in every document whose text contains any word starting with `s`. The user typed four characters and got the whole corpus back. - **Scoring is distorted.** BM25 weights terms by inverse document frequency, and one-character prefixes are the most common terms in the index. They contribute noise, and documents can outrank better ones simply by containing many short-prefix hits. - **Phrase and position semantics get strange.** The n-grams of a single word all sit at the same position, so phrase-style clauses behave in ways nobody intends. - **The query gets expensive.** Four to ten term lookups per typed word, each on a very long posting list, for what should be one cheap lookup. Raising `minimum_should_match` to 100% is a common but wrong-headed patch: it makes the extra terms harmless only because every prefix of the typed word is also a prefix of the stored word, so you are paying for redundant clauses that can never discriminate. Fixing the analyzer is strictly better. ## The correct configuration Declare the n-gram chain as the field's `analyzer` and a plain chain as its `search_analyzer`. The search analyzer must apply the *same normalization* — lowercasing, ASCII folding, whatever the index chain did before n-gramming — and simply stop short of the n-gram step. Then the typed `sta` becomes the single term `sta`, which the index already contains for every document whose text has a word starting with `sta`. ## The max_gram consequence Index-time n-gramming stores prefixes only up to `max_gram`. If `max_gram` is 10 and the user types 14 characters, the query term is 14 characters long and no stored term equals it, so a document that clearly starts with that string returns no hit. Two standard mitigations exist: truncate the query side (a `truncate` token filter in the search analyzer, sized to `max_gram`), or set `max_gram` high enough for realistic input at the cost of index size. Notice that this failure mode only exists *because* the sides are asymmetric — it is the price of the design, and interviewers like to see that you know the price rather than only the trick. ## Index cost and the alternatives Every document's terms are multiplied roughly by the average token length, so the field's postings grow substantially; keep the n-gram field as a dedicated sub-field or separate field rather than n-gramming everything, and do not enable it on large bodies of text. Alternatives with different trade-offs exist: `match_phrase_prefix` and `match_bool_prefix` do prefix expansion at query time with no index blow-up but more per-query work; the `search_as_you_type` field type packs a similar structure behind a single mapping declaration; a completion suggester serves a separate, sorted, in-memory structure optimized for suggestions rather than full search. The n-gram approach wins when you need fast prefix matching combined with ordinary relevance ranking and filtering over the same field. ## Diagnosing it in an interview The tell that a field is misconfigured is trivially checkable: run `_analyze` with the field, then with the search analyzer, on the same string. If the query side yields several short tokens rather than one, the field is n-gramming twice, and the symptom the team reported — "autocomplete returns irrelevant results as soon as you type one letter" — is fully explained.