Why does synonym_graph handle multi-word synonyms correctly where the older synonym filter does not?

## What a synonym filter actually does A synonym filter rewrites the token stream according to rules. In the default Solr-style format, `ny, new york` declares those forms equivalent, and `laptop => notebook, portable` maps explicitly in one direction. Rules can be inline under `synonyms`, or loaded from a file with `synonyms_path` relative to the node's config directory; recent 8.x versions also expose a synonyms API and a `synonyms_set` parameter so rule sets can be managed and reloaded without touching files on every node. The hard part is not the rules. It is positions. ## Positions and why they matter Every token carries a position. Phrase queries, `match_phrase` with a slop, and proximity scoring all work by comparing positions: `new` at position 3 and `york` at position 4 make an adjacent phrase. Position increments also encode alternatives — two tokens at the same position mean "either of these matches here". Single-word synonyms fit that model perfectly: replace `car` with `car` and `automobile` at the same position and everything downstream works. Multi-word synonyms do not. If the rule turns one token into two, the stream needs a branch that is *two positions long* running in parallel with a branch that is one position long. A flat token stream with position increments cannot express that faithfully. ## The old filter's failure mode The original `synonym` filter emits the replacement tokens with position increments that squeeze the multi-token alternative into the space of the original. The result is a stream where `york` may sit at the same position as the following real word, or where the injected phrase overlaps its neighbours. Symptoms in production: a phrase query for the synonym form matches documents it should not, or fails to match documents it should; slop values behave inconsistently; highlighting picks odd spans. The rules look right, the analysis output looks roughly right, and the search results are subtly wrong — which is why this is a senior-level question. ## The graph model `synonym_graph` emits a *graph* token stream: tokens carry both a position and a position length, so an alternative that spans two input positions is recorded as such. The query parser reads that graph and enumerates the valid paths through it, generating the correct set of phrase alternatives — effectively "either the single token `ny` here, or the two-token sequence `new york` covering the same span". Phrase and proximity semantics are then preserved for both forms. Other graph-producing filters exist for the same reason, notably `word_delimiter_graph`, which is the graph-aware replacement for the older `word_delimiter` filter. ## flatten_graph and the index-time constraint The index format stores positions, not graphs. So a graph stream cannot be indexed as-is. `synonym_graph` is therefore designed for the analyzer used at query time. If you do want the expansion baked into the index, the chain must end with `flatten_graph`, which collapses the graph into a flat stream — and in collapsing it, reintroduces exactly the multi-word imprecision the graph was designed to avoid. That trade-off, not the syntax, is the substance of the question. ## Rule parsing depends on the chain Synonym rules are text, and Elasticsearch analyzes them using the tokenizer and the filters that appear *before* the synonym filter. Two consequences follow. First, if a stemmer runs before the synonym filter, your rules are stemmed too, and a rule written as `running shoes` becomes something else — which may be what you want or may silently stop it matching. Second, some preceding filters are not usable for rule parsing, and Elasticsearch will either ignore them or reject the configuration depending on version; a chain that fails to create an index with a synonym filter in it is very often this. ## Operational notes `expand` (default `true`) controls how equivalence lists behave: with `expand: false`, `foo, bar, baz` is treated as though every form maps to the first. `lenient` lets the filter skip rules it cannot parse rather than failing outright — convenient for large hand-maintained files, dangerous because a typo then disappears silently. `format` accepts the default Solr syntax or `wordnet`. Large synonym sets are a real operational cost: rules are held per index and parsed at analyzer construction, so tens of thousands of rules affect memory and shard open time. Rules that expand aggressively also inflate query complexity, since every alternative path multiplies the clauses the query generates. ## The answer in one breath Multi-word synonyms need a branch of the token stream that is longer than one position; `synonym_graph` records that with position length and the query parser walks it, while the flat `synonym` filter cannot represent it and corrupts positions instead. Graph output is query-time by nature; indexing it requires `flatten_graph` and gives up some correctness.