What does Elasticsearch do with a keyword value longer than the field's ignore_above setting?

## The setting `ignore_above` is a mapping parameter on `keyword` fields. It takes a character count. When a document arrives whose value for that field is longer than the limit, Elasticsearch does not reject the document and does not truncate the value — it simply **does not add that value to the field's index structures**. Nothing goes into the inverted index, nothing goes into `doc_values`. For an array of strings, the rule applies per element: short elements are indexed, long ones are skipped, and the rest of the array is unaffected. ## Why it is so easy to miss The document is still indexed. It is still findable through every other field. And `_source` — the original JSON Elasticsearch stores for retrieval — is untouched, so when you fetch the document the long value is right there in the response. Everything looks normal. What breaks is structured access to that one field: - `{"term": {"url": "<the long value>"}}` returns nothing. - `{"exists": {"field": "url"}}` does not consider the document to have the field, because `exists` is driven by index structures. - Sorting on the field puts the document in the missing-value position. - A `terms` aggregation never produces a bucket for it, so counts under-report — and the total of the buckets no longer matches the document count. That combination — data visible in the hit, absent from the aggregation — is the diagnostic signature of `ignore_above`, and it is why the parameter shows up in interviews as a debugging scenario rather than a definition. ## Where the default comes from Most people never type `ignore_above`. They meet it because dynamic mapping applies it. When Elasticsearch maps a previously unseen string field, it creates a `text` field with a `keyword` sub-field carrying `ignore_above: 256`: ```json "url": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } ``` So a URL, a stack trace, a log message or a long product description is fully searchable through the analyzed parent and completely absent from `url.keyword` the moment it passes 256 characters. Aggregating on `url.keyword` then quietly reports on the short values only. ## Why the guard exists at all Lucene imposes a hard maximum on the byte length of a single indexed term. A `keyword` field with no limit that receives an arbitrarily long user-supplied string will hit it, and the indexing request fails with a document-level error. Setting `ignore_above` converts a hard failure into a silent skip, which is usually the better operational trade-off for machine-generated data of unpredictable length: one enormous log line should not fail an ingest batch. Note the units differ between the two: `ignore_above` counts characters, while Lucene's ceiling is in bytes after UTF-8 encoding. With multi-byte text, a value that passes a generous character limit can still exceed the byte ceiling, so leaving a comfortable margin matters for non-ASCII corpora. ## Getting it right Decide per field, from how the field is used: - **Identifiers, codes, host names, e-mails.** These have a known maximum. Set `ignore_above` a little above it so genuinely malformed input is dropped rather than blowing up the shard, and legitimate values are never lost. - **Long free text you also want to facet on exactly.** Reconsider the requirement. A faceted value that can be thousands of characters is rarely a useful bucket; either normalize it upstream (hash it, extract a category) or accept the analyzed field only. - **Fields you never filter, sort or aggregate on.** Drop the `keyword` sub-field entirely instead of tuning its limit. You save the storage and the confusion. ## Changing it later `ignore_above` can be updated on an existing mapping — it is one of the few mapping parameters that is not frozen. But like any mapping change it takes effect at index time, so documents already written with the old limit keep the old outcome: previously skipped values are not retroactively indexed. Raising the limit therefore requires rewriting the affected documents with `update_by_query` or a reindex before aggregations are trustworthy again. ## How to confirm it in a live system The cheap check is a mismatch test: run a query that returns the document by another field, confirm the value in `_source`, then run a `term` query on the exact value and an `exists` query on the field. If the document comes back from the first and not the others, and the mapping shows an `ignore_above` shorter than the value, you have your answer without touching the data.