Index-time vs Search-time Analysis

## The two moments Elasticsearch never matches raw strings. It matches **terms**, and terms are produced by an analyzer. Analysis happens at two distinct points in a document's life, and understanding which one you are looking at explains most "why does this query return nothing" bugs. **Index time.** When a document is indexed, every `text` field is passed through that field's analyzer. The analyzer is a three-stage pipeline: zero or more character filters (which rewrite the raw character stream, e.g. stripping HTML), exactly one tokenizer (which splits the stream into tokens and records their positions and offsets), and zero or more token filters (lowercasing, stemming, stopword removal, synonym expansion, ASCII folding). The tokens that come out the far end are written into the inverted index as terms, together with document ids, positions and frequencies. The original text is *not* stored in the index structure used for matching — it survives only in `_source` (for returning the document) and, if you enabled them, in `doc_values` or `stored` fields. **Search time.** When a full-text query such as `match`, `match_phrase`, `multi_match` or `query_string` targets that field, the query's input string is analyzed too, using the field's search analyzer. Each produced term becomes a lookup into the inverted index. `match` combines those lookups with a boolean OR by default; `match_phrase` additionally requires the terms to be adjacent according to the positions recorded at index time. ## Why compatibility matters Because matching happens purely between two sets of terms, the pipelines on the two sides must agree on the *shape* of a term. If the index-time analyzer lowercases and folds accents while the search-time one does not, the query term `Café` never finds the indexed term `cafe`. If the index-time analyzer stems (`running` → `run`) and the search side does not, searching for `running` misses documents that only ever produced `run`. The simplest way to guarantee agreement is what Elasticsearch does by default: unless you say otherwise, the same analyzer is used on both sides. You override the query side only with a deliberate purpose — the classic one being edge n-gram autocomplete, where you want many prefix terms in the index but only the whole typed word as the query term. ## Term-level queries are the exception `term`, `terms`, `prefix`, `wildcard` and `range` do **not** analyze their input. They take the string you supply and look it up verbatim. This is precisely correct for `keyword` fields, which are also not analyzed at index time (the whole value becomes one term), and it is the number-one beginner trap on `text` fields: `{"term": {"title": "Quick Brown"}}` on a standard-analyzed field finds nothing, because the index contains `quick` and `brown`, never `Quick Brown`. ## Seeing it for yourself The `_analyze` API is the diagnostic tool. Called with a `field`, it shows how that field's index-time analyzer tokenizes the text; called with an explicit `analyzer` name, it shows any pipeline you like. Comparing the two token lists side by side — the one your document produced and the one your query produces — resolves nearly every empty-result mystery in a minute. ## Practical consequences Two consequences follow, and interviewers usually push toward them: 1. **The index side is frozen into the data.** Terms were computed once, when the document was indexed. Changing the index-time analyzer does not rewrite existing segments; only newly indexed documents get the new terms. Making an analyzer change apply to old data means reindexing them. 2. **The search side is cheap to change.** The query analyzer runs per request, so swapping it — or reloading its synonym rules — takes effect immediately for all documents, old and new, with no reindex. That asymmetry is the whole reason the two sides can be configured separately, and it is why decisions like "expand synonyms at index time or at query time" have very different operational costs even though they can produce identical result sets.