skip to content

Query DSL

The JSON language you ask questions in: full-text match queries, exact term queries, and bool compositions that combine them. The recurring interview question is must vs filter — knowing which clauses score and which are cacheable is what separates a fast query from a slow one.

part ofElasticsearchoverview, primer and where to startread it →
on this pageshow

explore

questions

30

In an Elasticsearch bool query, what is the difference between a must clause and a filter clause?

level: juniorimportance: must knowfreq 85%

answer

  1. Both are mandatory clauses
  2. Only one of them touches ranking
  3. Query context versus filter context
  4. No score computed, result set reusable
  5. Filter results become cacheable per-segment bitsets

basics

~20 s

Both are mandatory and select exactly the same documents. A must clause runs in query context and adds a relevance contribution to _score; a filter clause runs in filter context, computes no score, and its result set can be reused from the node query cache.

solid answer

~40 s

They select the same documents — a hit must satisfy every `must` and every `filter` clause. The difference is the **context** the clause runs in. A `must` clause runs in *query context*: Elasticsearch asks "how well does this match?", computes a similarity contribution and sums it into `_score`. A `filter` clause runs in *filter context*: the question is only yes/no, so no score is computed, and the per-segment set of matching doc ids becomes a candidate for the node query cache — a repeated filter can then be answered from a cached bitset instead of re-executed. `must_not` is filter context too. In practice, put exact-value predicates — status, tenant id, tags, date ranges — in `filter`, and keep in `must` or `should` only the clauses whose *match quality* should influence ranking.

code

json · 11 lines
json
{
  "query": {
    "bool": {
      "must": [
        { "match": { "title": "wireless headphones" } },
        { "term": { "status": "published" } },
        { "range": { "created_at": { "gte": "now-30d/d" } } }
      ]
    }
  }
}

go deeper

for a junior

Be able to name the four bool slots and say which two skip scoring. Knowing that filter and must are both mandatory, and that filter is the default home for exact-value predicates, clears this screening question.

for a middle

Explain the mechanics: what work scoring actually does, why an unscored clause reduces to a doc-id set, and why that set can be cached per segment while a scored clause cannot.

for a senior

Show the diagnosis instinct — spotting a date range sitting in must on a hot endpoint, predicting that moving it changes score values but not ordering, and knowing filter-only bool queries return _score 0.0.

for a principal

Own the convention: a house query template where user text is the only scoring clause and every permission, tenancy and facet predicate is a filter, so filters repeat across users and stay cacheable.

## Two contexts, one query language Every leaf query in Elasticsearch's Query DSL runs in one of two contexts, and the context is decided by **where you put the clause**, not by which query type it is. The same `range` query is scored in one place and unscored in another. - **Query context** — the engine asks "how *well* does this document match?" It computes a relevance contribution (BM25 for full-text clauses, a flat constant for term-level ones) and sums it into the document's `_score`. - **Filter context** — the engine asks only "does this document match, yes or no?" No score is produced for the clause. The `bool` query is where most clauses acquire their context, which is why this distinction is usually taught as "must versus filter". ## The four slots of bool | Slot | Must the doc match? | Context | Affects `_score`? | |---|---|---|---| | `must` | yes | query | yes | | `should` | optional (see `minimum_should_match`) | query | yes | | `filter` | yes | filter | no | | `must_not` | must **not** match | filter | no | So `must` and `filter` are logically identical as *selectors*: AND semantics, mandatory. Swapping a clause between them never changes which documents come back. It changes only scoring and execution. ## Why filter context is cheaper Three separate savings stack up. **1. No score computation.** Scoring a term requires the term frequency in the document, the field-length norm, and the collection-wide document frequency, then a BM25 evaluation per matching document. A filter clause needs none of that — it only advances a doc-id iterator. For a clause matching millions of documents, that is real CPU. **2. Cacheability.** Because the answer is a pure doc-id set, Elasticsearch can hand it to the node query cache, which stores a per-segment bitset of matching doc ids. Segments are immutable, so a cached entry stays valid for the life of the segment; it is discarded when the segment is merged away. Query-context clauses cannot be cached this way, because their output is a score per document, not a set. **3. Better iteration.** With a conjunction of unscored clauses, Lucene leads with the cheapest, most selective iterator and skips ahead in the others, rather than fully evaluating every clause for every candidate. ## What actually changes in `_score` Moving a `range` or `term` clause from `must` to `filter` does change the absolute `_score` values, because a term-level query in query context contributes a constant (its `boost`, default 1.0) to the sum. But since every hit of the surrounding `bool` had to match that clause anyway, the constant was added to *every* hit — so the *relative ordering* is normally unchanged. That is exactly why the move is safe: you lose a meaningless constant, not ranking signal. One consequence to know: a `bool` query containing **only** `filter` and/or `must_not` clauses returns hits with `"_score": 0.0`, because nothing scored them. If you then sort by score you get an arbitrary order — supply an explicit `sort` instead. Also note that `boost` on a clause in filter context is ignored: there is no score to multiply. ## Choosing the slot Ask one question per clause: *should how well this matched change the ranking?* - "documents from this tenant" — no. `filter`. - "created in the last 30 days" — no, not by itself. `filter` (and if recency *should* affect ranking, that is a scoring function, not a range clause). - "status is published" — no. `filter`. - "the user's search phrase in title and body" — yes. `must` or `should`. - "exclude archived" — `must_not`. A common production shape is: exactly one scoring clause carrying the user's text, plus a stack of `filter` clauses carrying permissions, tenancy and facet selections. Those filters repeat across requests and users, so they are exactly the clauses that benefit most from caching. ## Diagnosing in the wild If a search endpoint is slower than it should be, look first at what sits in `must`. A date range or a high-cardinality `terms` clause in `must` is doing scoring work whose result is thrown away by the sort, and it is uncacheable. Moving it to `filter` is one of the cheapest wins available. Conversely, if someone reports "relevance broke when we refactored the query", check whether a genuinely discriminating full-text clause was demoted into `filter` — that clause *should* have been scoring. ## Pitfalls - Believing `filter` is "optional" or "a soft preference" — it is mandatory, like `must`. - Expecting a different result set after the move — the hit set is identical. - Assuming a filter is cached the first time it runs — caching is decided by a usage heuristic. - Setting `boost` on a filter clause and expecting it to do something.

  • If moving a range clause from must to filter does not change which documents match, why do the _score values change?
    A term-level query in query context still contributes a constant to the score — its boost, 1.0 by default. Every hit of the bool had to match that clause, so every hit received the same constant. Removing it lowers all scores by the same amount, which is why the ranking order is normally identical even though the numbers differ.
  • What _score do documents get from a bool query that contains only filter clauses, and why does that matter?
    They come back with _score of 0.0, because no clause ran in query context. It matters if the request relies on the default sort by score: with every hit tied at zero the order is effectively arbitrary. Add an explicit sort on a doc_values-backed field, or wrap a clause in constant_score if you want a non-zero flat score.
  • Does must_not run in query context or filter context?
    Filter context. A must_not clause only excludes documents; it contributes nothing to _score and, like filter, its doc-id set is a candidate for the node query cache. A candidate who claims must_not subtracts from the score has the model wrong — negative scoring is what the boosting query's negative_boost is for.

must is a graded exam question: you have to answer it and the answer affects your mark. filter is a door policy: you either meet it or you are not in the room, and meeting it earns you no extra points.

saying these in an interview costs you the question

  • Says filter clauses are optional or merely preferences
  • Claims filter returns a different document set than must
  • Thinks must_not subtracts from _score
  • Believes every filter clause is cached on its first execution
  • Sets boost on a filter clause expecting a ranking change

context

open as a page

In Elasticsearch, what does a match query do with the query string before it looks up terms?

level: juniorimportance: must knowfreq 82%

basics

~10 s

A match query runs the string through the field's search analyzer, turning it into terms, then builds a boolean query over those terms. By default a document matching any single term is a hit.

open as a page

Why does Elasticsearch reject a search with from=10000 and size=10?

level: juniorimportance: must knowfreq 70%

basics

~20 s

Because from + size exceeds index.max_result_window, which defaults to 10,000. Deep paging makes every shard build a top-(from+size) list that the coordinating node merges and then throws almost all of away, so Elasticsearch caps the depth instead.

open as a page

Why does an Elasticsearch term query on a text field usually return no hits?

level: juniorimportance: must knowfreq 88%

basics

~20 s

A term query looks up the exact bytes you supply with no analysis, while a text field stores analyzed tokens that are split and lowercased. Searching for New York finds no such token. Use match, or a keyword sub-field.

open as a page

When do should clauses in an Elasticsearch bool query affect which documents match?

level: middleimportance: must knowfreq 70%

basics

~20 s

Only when minimum_should_match requires it. If the bool query has at least one should clause and no must or filter clauses, minimum_should_match defaults to 1; otherwise it defaults to 0, so should clauses only add to _score.

open as a page

How does Elasticsearch's nested field type change the way an array of objects is indexed and queried?

level: middleimportance: must knowfreq 72%

basics

~20 s

Mapping a field as nested indexes each object in the array as its own hidden Lucene document, stored in one block with its parent. A nested query with a path then requires all clauses to match the same inner object.

open as a page

Why does an Elasticsearch bool query on an array of objects match fields from different objects?

level: middleimportance: must knowfreq 78%

basics

~20 s

By default Elasticsearch flattens an array of objects into independent multi-valued fields, losing which value belonged to which entry. A query for name alice and age 25 then matches even when those values came from different entries.

open as a page

How do you page beyond 10,000 hits in Elasticsearch using search_after and a point in time?

level: middleimportance: must knowfreq 70%

basics

~20 s

Open a point in time, sort by a field plus a unique tiebreaker, then send each next page with search_after set to the previous page's last hit sort array. Cost per page stays constant, and the PIT keeps pages consistent while indexing continues.

open as a page

How does date math like now-7d/d work in an Elasticsearch range query?

level: middleimportance: must knowfreq 62%

basics

~20 s

Date math takes an anchor (now, or a date followed by two pipes), applies offsets such as minus seven days, then optionally rounds with a slash plus a unit. Rounding direction depends on which endpoint uses it.

open as a page

How do the multi_match types best_fields, most_fields and cross_fields differ in Elasticsearch?

level: seniorimportance: must knowfreq 64%

basics

~20 s

best_fields takes the best-scoring single field, so all query terms should be in one field. most_fields sums the scores of several analyses of the same text. cross_fields is term-centric: it treats the listed fields as one big field so terms may be spread across them.

open as a page

Which field values does Elasticsearch's exists query treat as missing?

level: juniorimportance: should knowfreq 42%

basics

~20 s

The exists query matches documents that have an indexed value for a field. A JSON null, an empty array, and an array of only nulls all count as missing, as do values skipped by ignore_above or ignore_malformed.

open as a page

What does wrapping a query in Elasticsearch's constant_score query do to matching and scoring?

level: middleimportance: should knowfreq 45%

basics

~20 s

constant_score runs its inner query in filter context and gives every matching document the same score, taken from its boost parameter, which defaults to 1.0. Matching is unchanged; the inner query's own relevance calculation is discarded.

open as a page

In an Elasticsearch dis_max query, what does the tie_breaker parameter change?

level: middleimportance: should knowfreq 40%

basics

~20 s

dis_max scores a document by its single best-matching sub-query. tie_breaker, default 0.0, adds that fraction of every other matching sub-query's score to the total, so documents matching several clauses edge ahead of equally-good single-clause matches.

open as a page

How does the fuzziness parameter behave in an Elasticsearch match query, and what does AUTO mean?

level: middleimportance: should knowfreq 55%

basics

~20 s

fuzziness allows each analyzed term to match index terms within an edit distance, capped at 2 edits. AUTO scales with term length: no edits for terms of 1-2 characters, one edit for 3-5, two edits above that.

open as a page

What must a document contain to match an Elasticsearch match_phrase query, and what does slop change?

level: middleimportance: should knowfreq 70%

basics

~20 s

match_phrase requires all analyzed terms in the same field, in the given order, at consecutive positions. slop is the number of position moves allowed, so a non-zero slop tolerates intervening words and, at a cost of two moves, reversed pairs.

open as a page

In an Elasticsearch match query, how do operator and minimum_should_match change which documents match?

level: middleimportance: should knowfreq 66%

basics

~10 s

Both control how many of the analyzed terms a document must contain. operator switches between any term (or) and every term (and); minimum_should_match sets a count or percentage in between, trading recall for precision.

open as a page

What does inner_hits add to an Elasticsearch nested query's results?

level: middleimportance: should knowfreq 48%

basics

~20 s

inner_hits attaches to each matching document the specific nested or child documents that caused the match, with their own source, scores, sorting and highlighting. Without it the hit's _source contains every child and you cannot tell which one matched.

open as a page

How does Elasticsearch's join field work with has_child and has_parent queries?

level: middleimportance: should knowfreq 55%

basics

~10 s

A join field declares parent-child relations inside one index. Children are independent documents that must be routed to their parent's shard; has_child returns parents whose children match, and has_parent returns children whose parent matches.

open as a page

Why does sorting an Elasticsearch search on a text field fail, and what do you sort on instead?

level: middleimportance: should knowfreq 62%

basics

~20 s

Sorting needs one value per document read from a columnar doc_values structure, and analyzed text fields have doc_values disabled, so the request errors telling you fielddata is off. Sort on a keyword sub-field such as title.keyword, which has doc_values by default.

open as a page

Why does hits.total report 10000 with relation gte, and what does track_total_hits change?

level: middleimportance: should knowfreq 58%

basics

~20 s

Elasticsearch stops counting matches accurately at 10,000 by default so top-k queries can skip non-competitive documents. A relation of gte means at least that many matched. track_total_hits: true forces an exact count, false skips counting entirely, and an integer sets a different threshold.

open as a page

What does fuzziness AUTO mean in an Elasticsearch fuzzy query?

level: middleimportance: should knowfreq 40%

basics

~10 s

AUTO scales the allowed edit distance with term length: no edits for very short terms, one edit for medium ones, two for longer ones. Edit distance counts insertions, deletions, substitutions, and by default transpositions.

open as a page

When would you use terms_lookup instead of inlining values in a terms query?

level: middleimportance: should knowfreq 40%

basics

~20 s

Use terms_lookup when the list of values already lives in another document — a user's group memberships or blocked ids — so Elasticsearch fetches it server-side instead of your application shipping thousands of values in every request.

open as a page

Which parts of an Elasticsearch bool query can the node query cache accelerate?

level: seniorimportance: should knowfreq 42%

basics

~20 s

Only clauses running in filter context — the filter and must_not slots, plus anything inside constant_score. The cache stores per-segment doc-id sets for those clauses, so a repeated filter is answered from a bitset instead of re-executed. Scored clauses are never cached this way.

open as a page

When would you expose query_string versus simple_query_string to end users in Elasticsearch?

level: seniorimportance: should knowfreq 50%

basics

~20 s

Prefer simple_query_string for anything user-facing: it never fails on bad syntax and its operator set can be restricted. Reserve query_string, which parses full Lucene syntax and returns an error on malformed input, for trusted power users.

open as a page

What does mapping a field as nested in Elasticsearch cost at index and update time?

level: seniorimportance: should knowfreq 52%

basics

~20 s

Every nested object becomes its own Lucene document, so a document with 200 entries indexes as 201. Any update rewrites the whole block, deleting and re-adding every child, and dedicated limits cap nested fields per index and nested objects per document.

open as a page

When should you still use Elasticsearch's scroll API instead of search_after with a point in time?

level: seniorimportance: should knowfreq 45%

basics

~20 s

Almost never for new code. Scroll is a stateful snapshot cursor built for exporting an entire result set once, and Elasticsearch no longer recommends it for deep pagination; search_after with a point in time covers the same ground statelessly. Scroll survives mainly in existing batch jobs.

open as a page

An Elasticsearch search is fast until it returns 100 large documents per page — how do you cut the fetch cost?

level: seniorimportance: should knowfreq 45%

basics

~20 s

That cost is the fetch phase reading and decompressing _source for every returned hit. Return fewer hits per page, use _source filtering to shrink the payload, read small values from doc_values with docvalue_fields, or map the few needed fields with store true so _source is never touched.

open as a page

Why is an Elasticsearch wildcard query like *smith slow, and what replaces it?

level: seniorimportance: should knowfreq 52%

basics

~20 s

A leading wildcard gives no fixed prefix, so Elasticsearch must walk every term in that field's dictionary and test each one. The fix is index-time: index_prefixes, an edge n-gram or reversed field, or the wildcard field type.

open as a page

In Elasticsearch, how do you choose between nested, a join field, and denormalization?

level: principalimportance: should knowfreq 36%

basics

~20 s

Start from denormalization: one document per entity you want to return. Add nested only when the many-side is small, bounded and updated with its parent. Use a join field only when children churn far more often than parents and query volume can absorb it.

open as a page

How does Elasticsearch's boosting query demote documents compared with excluding them via must_not?

level: seniorimportance: nice to knowfreq 30%

basics

~20 s

must_not removes matching documents from the result set entirely. The boosting query keeps them: documents matching its positive query are returned, and those also matching its negative query have their score multiplied by negative_boost, so they sink rather than disappear.

open as a page