When is the Elasticsearch flattened field type the right choice for an object with unpredictable keys?

## The problem it solves Some documents carry an object whose keys are not under your control: labels attached by users, per-tenant custom attributes, arbitrary tags from an ingestion pipeline. With ordinary `object` mapping, every new key that appears becomes a new field in the index mapping. Across a large corpus that grows without bound — every distinct key from every distinct tenant is a permanent entry in the mapping, the cluster state that carries it grows, and eventually indexing fails when the index's field limit is reached. This runaway growth is the classic mapping-explosion failure, and it is what `flattened` exists to prevent. ## What flattened does Declaring a field as `flattened` maps the *whole subtree* as one field: ```json "attributes": { "type": "flattened" } ``` At index time Elasticsearch walks the object and indexes each leaf value as a keyword, keeping track of which key it came from. The mapping never grows: no matter how many distinct keys the documents contain, the mapping still shows exactly one field. You then query in two ways. Against the field itself, a value matches if it appears **anywhere** in the object — useful for "does this document have this value at all?" Against a dotted path, the value must appear under that specific key: ```json { "query": { "term": { "attributes.color": "red" } } } { "query": { "term": { "attributes": "red" } } } ``` Doc values are maintained, so `terms` aggregations and sorting on a sub-key work, which is what makes the type genuinely useful for faceting over open-ended metadata. ## What you give up The simplification is not free, and articulating the cost is what separates a real answer from a memorized definition. - **Everything is a string.** A leaf value of `42` is indexed as the keyword `"42"`. A range query on it compares strings lexicographically, so `"9"` sorts after `"42"`. Dates are strings too, so no date math and no date histograms. - **No analysis.** Values are not tokenized, lowercased or stemmed, so full-text search inside a flattened object does not work the way a `text` field does. Matching is exact-term matching. - **Limited query surface.** Only a subset of query types is supported against flattened fields. Anything expecting a richly typed field — and any feature that needs per-field analysis — is unavailable. - **Flat relevance.** Because there are no per-field statistics of the usual kind, scoring inside a flattened field is not a useful ranking signal. Treat it as a filter surface, not a relevance surface. - **Object structure is not preserved for correlation.** Like ordinary object mapping, an array of sub-objects loses the association between keys within one element, so you cannot ask "the element whose colour is red *and* whose size is large". - **Depth is bounded.** There is a configurable limit on how deeply the object may nest, so pathologically deep structures still fail rather than being indexed silently. ## Choosing between the options Given an object with many keys, you have four realistic choices, and the interview answer is knowing when each applies: 1. **Explicit `object` mapping** with `dynamic: strict` — when the key set is known and stable. Best typing, best relevance, no surprises; requires you to own the schema. 2. **`flattened`** — when the keys are open-ended and you only filter and facet on the values. Constant mapping cost, string-only semantics. 3. **Key/value pair array** — model `[{"key": "color", "value": "red"}]` with typed `key` and `value` fields. Keeps proper types and, with the appropriate mapping, keeps key and value correlated, at the cost of clumsier queries. 4. **Store as an unindexed blob** — if you only display the metadata, keep it in `_source` and index nothing. ## A practical shape Most production designs are hybrids. Promote the handful of attributes that matter — the ones you range over, rank on, or search as text — into explicit, properly typed fields, and let `flattened` absorb the long tail nobody queries by name. That gives you correct semantics where they are needed and a bounded mapping everywhere else. One last operational note: because the field type cannot be changed in place, the choice between `object` and `flattened` is effectively permanent for a given index. On a data stream or a rolled-over time-series index that hurts less — new backing indices pick up the corrected template — but for a long-lived index it means a reindex. Decide before the first ingest, not after the field-limit error arrives during an incident.