skip to content

In an Elasticsearch bool query, what is the difference between a must clause and a filter clause?

level: juniorimportance: must knowfreq 85%

answer

  1. Both are mandatory clauses
  2. Only one of them touches ranking
  3. Query context versus filter context
  4. No score computed, result set reusable
  5. Filter results become cacheable per-segment bitsets

basics

~20 s

Both are mandatory and select exactly the same documents. A must clause runs in query context and adds a relevance contribution to _score; a filter clause runs in filter context, computes no score, and its result set can be reused from the node query cache.

solid answer

~40 s

They select the same documents — a hit must satisfy every `must` and every `filter` clause. The difference is the **context** the clause runs in. A `must` clause runs in *query context*: Elasticsearch asks "how well does this match?", computes a similarity contribution and sums it into `_score`. A `filter` clause runs in *filter context*: the question is only yes/no, so no score is computed, and the per-segment set of matching doc ids becomes a candidate for the node query cache — a repeated filter can then be answered from a cached bitset instead of re-executed. `must_not` is filter context too. In practice, put exact-value predicates — status, tenant id, tags, date ranges — in `filter`, and keep in `must` or `should` only the clauses whose *match quality* should influence ranking.

code

json · 11 lines
json
{
  "query": {
    "bool": {
      "must": [
        { "match": { "title": "wireless headphones" } },
        { "term": { "status": "published" } },
        { "range": { "created_at": { "gte": "now-30d/d" } } }
      ]
    }
  }
}

go deeper

for a junior

Be able to name the four bool slots and say which two skip scoring. Knowing that filter and must are both mandatory, and that filter is the default home for exact-value predicates, clears this screening question.

for a middle

Explain the mechanics: what work scoring actually does, why an unscored clause reduces to a doc-id set, and why that set can be cached per segment while a scored clause cannot.

for a senior

Show the diagnosis instinct — spotting a date range sitting in must on a hot endpoint, predicting that moving it changes score values but not ordering, and knowing filter-only bool queries return _score 0.0.

for a principal

Own the convention: a house query template where user text is the only scoring clause and every permission, tenancy and facet predicate is a filter, so filters repeat across users and stay cacheable.

## Two contexts, one query language Every leaf query in Elasticsearch's Query DSL runs in one of two contexts, and the context is decided by **where you put the clause**, not by which query type it is. The same `range` query is scored in one place and unscored in another. - **Query context** — the engine asks "how *well* does this document match?" It computes a relevance contribution (BM25 for full-text clauses, a flat constant for term-level ones) and sums it into the document's `_score`. - **Filter context** — the engine asks only "does this document match, yes or no?" No score is produced for the clause. The `bool` query is where most clauses acquire their context, which is why this distinction is usually taught as "must versus filter". ## The four slots of bool | Slot | Must the doc match? | Context | Affects `_score`? | |---|---|---|---| | `must` | yes | query | yes | | `should` | optional (see `minimum_should_match`) | query | yes | | `filter` | yes | filter | no | | `must_not` | must **not** match | filter | no | So `must` and `filter` are logically identical as *selectors*: AND semantics, mandatory. Swapping a clause between them never changes which documents come back. It changes only scoring and execution. ## Why filter context is cheaper Three separate savings stack up. **1. No score computation.** Scoring a term requires the term frequency in the document, the field-length norm, and the collection-wide document frequency, then a BM25 evaluation per matching document. A filter clause needs none of that — it only advances a doc-id iterator. For a clause matching millions of documents, that is real CPU. **2. Cacheability.** Because the answer is a pure doc-id set, Elasticsearch can hand it to the node query cache, which stores a per-segment bitset of matching doc ids. Segments are immutable, so a cached entry stays valid for the life of the segment; it is discarded when the segment is merged away. Query-context clauses cannot be cached this way, because their output is a score per document, not a set. **3. Better iteration.** With a conjunction of unscored clauses, Lucene leads with the cheapest, most selective iterator and skips ahead in the others, rather than fully evaluating every clause for every candidate. ## What actually changes in `_score` Moving a `range` or `term` clause from `must` to `filter` does change the absolute `_score` values, because a term-level query in query context contributes a constant (its `boost`, default 1.0) to the sum. But since every hit of the surrounding `bool` had to match that clause anyway, the constant was added to *every* hit — so the *relative ordering* is normally unchanged. That is exactly why the move is safe: you lose a meaningless constant, not ranking signal. One consequence to know: a `bool` query containing **only** `filter` and/or `must_not` clauses returns hits with `"_score": 0.0`, because nothing scored them. If you then sort by score you get an arbitrary order — supply an explicit `sort` instead. Also note that `boost` on a clause in filter context is ignored: there is no score to multiply. ## Choosing the slot Ask one question per clause: *should how well this matched change the ranking?* - "documents from this tenant" — no. `filter`. - "created in the last 30 days" — no, not by itself. `filter` (and if recency *should* affect ranking, that is a scoring function, not a range clause). - "status is published" — no. `filter`. - "the user's search phrase in title and body" — yes. `must` or `should`. - "exclude archived" — `must_not`. A common production shape is: exactly one scoring clause carrying the user's text, plus a stack of `filter` clauses carrying permissions, tenancy and facet selections. Those filters repeat across requests and users, so they are exactly the clauses that benefit most from caching. ## Diagnosing in the wild If a search endpoint is slower than it should be, look first at what sits in `must`. A date range or a high-cardinality `terms` clause in `must` is doing scoring work whose result is thrown away by the sort, and it is uncacheable. Moving it to `filter` is one of the cheapest wins available. Conversely, if someone reports "relevance broke when we refactored the query", check whether a genuinely discriminating full-text clause was demoted into `filter` — that clause *should* have been scoring. ## Pitfalls - Believing `filter` is "optional" or "a soft preference" — it is mandatory, like `must`. - Expecting a different result set after the move — the hit set is identical. - Assuming a filter is cached the first time it runs — caching is decided by a usage heuristic. - Setting `boost` on a filter clause and expecting it to do something.

  • If moving a range clause from must to filter does not change which documents match, why do the _score values change?
    A term-level query in query context still contributes a constant to the score — its boost, 1.0 by default. Every hit of the bool had to match that clause, so every hit received the same constant. Removing it lowers all scores by the same amount, which is why the ranking order is normally identical even though the numbers differ.
  • What _score do documents get from a bool query that contains only filter clauses, and why does that matter?
    They come back with _score of 0.0, because no clause ran in query context. It matters if the request relies on the default sort by score: with every hit tied at zero the order is effectively arbitrary. Add an explicit sort on a doc_values-backed field, or wrap a clause in constant_score if you want a non-zero flat score.
  • Does must_not run in query context or filter context?
    Filter context. A must_not clause only excludes documents; it contributes nothing to _score and, like filter, its doc-id set is a candidate for the node query cache. A candidate who claims must_not subtracts from the score has the model wrong — negative scoring is what the boosting query's negative_boost is for.

must is a graded exam question: you have to answer it and the answer affects your mark. filter is a door policy: you either meet it or you are not in the room, and meeting it earns you no extra points.

saying these in an interview costs you the question

  • Says filter clauses are optional or merely preferences
  • Claims filter returns a different document set than must
  • Thinks must_not subtracts from _score
  • Believes every filter clause is cached on its first execution
  • Sets boost on a filter clause expecting a ranking change

context

resolving links…