Why does an Elasticsearch bool query on an array of objects match fields from different objects?

## The setup Consider an index that leaves `users` as an ordinary object field — the default when Elasticsearch first sees an array of JSON objects — and a document: ```json { "group": "fans", "users": [ {"name": "alice", "age": 30}, {"name": "john", "age": 25} ] } ``` A `bool` query with two `must` clauses, `term users.name: alice` and `term users.age: 25`, matches this document even though no single user is both alice and 25. ## What Lucene actually stores Lucene has no concept of structure inside a document. A Lucene document is a flat set of field-to-values pairs. When Elasticsearch indexes an array of objects under an `object` field, it walks the JSON tree and appends every leaf value to the field named by its full dotted path. The document above becomes, internally, the equivalent of: ``` users.name: ["alice", "john"] users.age: [30, 25] ``` Two independent multi-valued fields. Each field remembers its own values — and, for analyzed text, the token positions inside that field — but nothing records that `alice` and `30` arrived in the same JSON object. Array boundaries are gone the moment the document is indexed. `_source` still holds the original JSON, which is why the API response looks perfectly structured, but `_source` is a stored blob that is returned after matching; it is not what queries run against. ## Why the query matches Query clauses are predicates over a document, not over an array element. `term users.name: alice` asks whether this document's `users.name` field contains the term alice — it does. `term users.age: 25` asks the same of `users.age` — it does too. `bool.must` requires both predicates to hold for the same document, and both hold. The more entries the array carries, the more spurious combinations become matchable, so a filter that reads as precise silently over-matches. The same flattening distorts analytics: a `terms` aggregation on `users.name` with an `avg` sub-aggregation on `users.age` averages *all* the document's ages inside *every* name bucket, because at document level every name is effectively paired with every age. ## When flattening is harmless Flattening only loses information when an array has more than one entry **and** a query correlates two or more fields from it. If the object always holds a single entry — an `address` with a `city` and a `zip` on a one-address document — or if queries only ever touch one field at a time, the flattened form answers correctly and costs far less than the alternatives. That is exactly why `object` is the default: most fields never need the extra machinery, and paying for it everywhere would be wasteful. ## The fixes, in the order to consider them **Denormalize.** Index one document per array entry — one document per user, carrying the group's fields alongside. Correlation becomes trivial because a single document now holds exactly one name and one age; queries are plain and fast and need no special syntax. The costs are duplicated parent fields, re-indexing every child when a parent field changes, and needing a `terms` aggregation on a group id (or a second query) to roll results back up to the group. **Map the field as `nested`.** This tells Elasticsearch to index each object in the array as its own hidden Lucene document, preserving the association, so a `nested` query can demand that all clauses match the *same* inner object. It is the direct fix, but it multiplies the physical document count and forces a full rewrite of the document and all of its children on any update. **Fake the correlation with a composite key.** When exactly one combination matters and it is a filter rather than open-ended search, index a synthetic keyword such as `users.name_age: ["alice|30", "john|25"]` and query it with one `term`. One field, one term, no ambiguity, none of the nested cost — but it only answers the combinations you anticipated at index time. ## What does not fix it The `flattened` field type does not preserve object identity either; it maps a whole subtree into a single field of key/value pairs so unpredictable keys cannot explode the mapping, and it is subject to the same cross-object matching. Adding more `must` clauses, tightening `minimum_should_match`, or switching from a `text` field to its `keyword` sub-field changes which terms match but never restores the lost association. ## Recognising it in the wild The signature is a filter returning documents that visibly fail it when you read `_source`, and only for documents whose arrays hold more than one entry. Reproduce it deliberately: index a two-entry document and query an impossible combination. If the document comes back, the field is flattened.