Which field values does Elasticsearch's exists query treat as missing?

## What exists actually asks The `exists` query takes a single `field` parameter and matches documents for which Elasticsearch has an indexed value for that field. The important word is *indexed*. It is not asking whether the key appears in the JSON you sent, and it is not reading `_source`. That distinction is the source of every surprise this query produces. ## The values that count as missing A document does **not** match `exists` when: - **The field is absent** from the indexed document entirely. - **The value is `null`.** JSON null is explicitly "no value" and nothing is indexed for it. - **The value is an empty array `[]`**, or an array containing only nulls such as `[null, null]`. An array is indexed as its elements; with no non-null elements there is nothing to index. - **The value exceeded `ignore_above`** on a `keyword` field. That mapping parameter tells Elasticsearch to skip indexing values longer than N characters. Nothing is indexed, so `exists` does not see it — even though the full string is returned in `_source`. This is the most confusing case in practice, because the dynamically created `.keyword` sub-field of a string carries `ignore_above: 256`, so long values quietly fall out of exact-match queries *and* `exists`. - **The value was malformed and `ignore_malformed` is set.** Sending `"abc"` to a field mapped as `integer` normally fails the whole document; with `ignore_malformed: true` the document is indexed with that one field skipped. Again `_source` keeps the bad value while the index has nothing. And the case that surprises people the other way: **an empty string is a value**. `{"comment": ""}` on a `keyword` field indexes a zero-length term, and `exists` matches. If your definition of "has a comment" excludes the empty string, `exists` alone will not express it — you need an additional clause. ## Finding the documents without a value There is no `missing` query in modern Elasticsearch; it was removed and the idiom is negation: ```json { "query": { "bool": { "must_not": { "exists": { "field": "user.id" } } } } } ``` This is a filter-shaped clause — it does not contribute to relevance — so it belongs in `filter` or `must_not` rather than `must`, and it is a good candidate for caching when reused. ## The null_value escape hatch If you want nulls to be searchable, map the field with `null_value`. On a `keyword` field, `"null_value": "NULL"` causes an explicit JSON `null` to be indexed as the term `NULL`. Two consequences follow: those documents now match `exists`, and you can search for them explicitly with a term query for `NULL`. The parameter's type must match the field's — a string for `keyword`, a number for a numeric field — and it substitutes only for explicit nulls, not for an absent field. It is the right tool when "the caller explicitly said none" is semantically different from "the caller said nothing", such as an optional attribute the user actively cleared. ## Multi-fields and object paths `exists` on `title` and `exists` on `title.keyword` can disagree, because they are different indexed fields with different mapping parameters — the `ignore_above` case above is exactly that. Name the sub-field you actually query. On object fields, `exists` with a parent path such as `user` matches documents having an indexed value for any leaf under it, since the object itself is not a field but a namespace of `user.id`, `user.name` and so on. ## Where it shows up in real work The most common production use is data-quality auditing: counting how many documents in an index lack a field you thought was mandatory, usually right after a mapping or pipeline change. The second is optional-filter logic — "only items that have a discount" — where a naive implementation checks `_source` in the application and disagrees with what search returns. When those two disagree, the answer is almost always `ignore_above`, `ignore_malformed`, or a field mapped with `index: false`, and the fix is to reconcile the mapping with what the application believes, not to work around it at query time.