skip to content

How do dynamic_templates control the types Elasticsearch assigns to newly seen fields?

level: middleimportance: should knowfreq 48%

answer

  1. An ordered list, not a set of independent rules
  2. Matching happens on shape, not on known names
  3. One predicate looks at the leaf name, one at the path
  4. First match wins, so ordering is the bug
  5. Placeholders exist for name and detected type

basics

~20 s

dynamic_templates is an ordered list of rules in an Elasticsearch mapping. Each rule matches unmapped fields by detected JSON type, field name, or dotted path, and supplies the mapping to use. The first matching rule wins.

solid answer

~50 s

`dynamic_templates` lets you keep dynamic mapping's convenience while dictating the result. It is an **ordered array** of single-key rules in the mapping; when an unmapped field appears, Elasticsearch walks the list and applies the **first** rule whose predicates all match. Predicates are `match_mapping_type` (the type dynamic mapping inferred: `string`, `long`, `double`, `boolean`, `date`, `object`, or `*`), `match`/`unmatch` on the field's own name, and `path_match`/`path_unmatch` on its full dotted path; wildcards are the default, and `match_pattern: "regex"` switches to regular expressions. The `mapping` block is the definition to use, and it may interpolate `{name}` and `{dynamic_type}`. The classic rule maps every detected string as a plain `keyword` instead of `text` plus a `keyword` sub-field — that fixes types for log-style data and halves the number of mapping entries. Templates only apply to fields not already in the mapping.

code

json · 21 lines
json
PUT /events
{
  "mappings": {
    "dynamic_templates": [
      { "ids_as_keyword": {
          "match": "*_id",
          "match_mapping_type": "string",
          "mapping": { "type": "keyword" }
      }},
      { "labels_not_indexed": {
          "path_match": "labels.*",
          "match_mapping_type": "*",
          "mapping": { "type": "{dynamic_type}", "index": false }
      }},
      { "strings_as_keyword": {
          "match_mapping_type": "string",
          "mapping": { "type": "keyword", "ignore_above": 256 }
      }}
    ]
  }
}

go deeper

for a junior

Know that dynamic_templates exist and that the common one maps detected strings to keyword instead of text plus a keyword sub-field. Recognising the syntax in a mapping is enough here.

for a middle

Explain the predicates — match_mapping_type, match/unmatch, path_match/path_unmatch — that first match wins, and that the mapping block can use the {name} and {dynamic_type} placeholders.

for a senior

Show how you would roll templates out through index templates so new indices are born correct, and explain why they fix field types without bounding field count. Be ready to debug a template that never fires.

for a principal

Treat these as the schema contract for teams that ship unforeseeable fields: what house rules the platform imposes centrally, how they are versioned, and what the migration story is for indices created before them.

## The problem they solve Explicit mappings are exact but require you to know every field name in advance. Dynamic mapping needs no foreknowledge but produces whatever its inference rules produce. `dynamic_templates` sit in between: you cannot name the fields, but you can state a rule about *shape* — "every string under `labels.*` is a keyword", "every double is stored but not indexed" — and have it applied to fields you have never seen. ## Structure The mapping holds an **array**, and each element is an object with exactly one key: the template's name, which exists only for readability and for the error messages. ```json "dynamic_templates": [ { "strings_as_keyword": { "match_mapping_type": "string", "mapping": { "type": "keyword", "ignore_above": 256 } }}, { "metrics_as_double": { "path_match": "metrics.*", "match_mapping_type": "long", "mapping": { "type": "double" } }} ] ``` Order matters and is the most common source of confusion: evaluation stops at the first template whose predicates all match, so a broad `match_mapping_type: "*"` rule placed first makes everything after it dead code. Put the specific rules first. ## The predicates - **`match_mapping_type`** matches the type dynamic detection *would* have chosen: `string`, `long`, `double`, `boolean`, `date`, `object`, or `*` for any. Note that this is the detected JSON-level type, so an unquoted whole number is `long` and a fractional one is `double`, regardless of what you want the final field to be. - **`match` / `unmatch`** test the field's own (leaf) name, with `*` wildcards — `"match": "*_id"`. - **`path_match` / `path_unmatch`** test the full dotted path, so `"path_match": "user.social.*"` targets a subtree rather than a name pattern. - **`match_pattern: "regex"`** switches `match`/`unmatch` from glob to Java regular expressions. When several predicates are present they are ANDed. Predicates can also be supplied as arrays of patterns to match any of several shapes. ## The mapping block and its placeholders The `mapping` value is an ordinary field definition — anything you could have written under `properties`, including multi-fields, `index: false`, `ignore_above`, `null_value`, analyzers, and so on. Two placeholders are substituted: - `{name}` — the field's name, useful for `"copy_to": "{name}_all"`-style constructions and for naming analyzers per field; - `{dynamic_type}` — the detected type, which lets one template say "whatever type you detected, but with doc values off". A template may also declare a `runtime` block instead of `mapping`, mapping matching fields as runtime fields rather than indexed ones — a middle ground between `dynamic: true` and `dynamic: runtime` that applies only to the shapes you choose. ## What they are used for in practice **Killing the text/keyword pair.** For logs and metrics you almost never need full-text analysis on arbitrary string fields, and each such field otherwise costs two mapping entries plus an analysed index. One `match_mapping_type: "string" → keyword` template is the single highest-value template most clusters have. **Type correction by convention.** `*_id` as `keyword`, `*_at` as `date`, `is_*` as `boolean` — a naming convention becomes a schema without enumerating fields. **Turning off what you do not query.** `index: false` on a bulky subtree keeps values retrievable through `_source` while cutting index size; `doc_values: false` on fields you never sort or aggregate saves disk. ## Limits worth knowing Templates apply only to fields that are **not** already mapped, and only when the effective `dynamic` value permits new fields at all — with `dynamic: strict` nothing is dynamically added, so no template will ever fire. They do not retroactively change fields that were mapped before the template existed; that still means a reindex. And they still do not cap *how many* fields appear — a template can dictate that every new key is a keyword while the number of keys grows without bound, which is a separate problem with a separate control. Because they are part of the mapping, templates are normally shipped in an index template so every new index (or data-stream backing index) is born with the house rules already in place, rather than being retrofitted after the first bad field has landed.

  • Why is putting a match_mapping_type of "*" template first a bug?
    Because evaluation stops at the first template whose predicates match. A `"*"` rule matches every unmapped field, so every template listed after it is unreachable. Order specific rules — name patterns, path subtrees, particular detected types — before broad catch-alls, and treat the list as a rule chain rather than an unordered set.
  • What is the difference between match and path_match in a dynamic template?
    `match` tests only the field's own leaf name, so `"match": "count"` fires for `count` anywhere in the document, including `metrics.count`. `path_match` tests the full dotted path from the document root, so `"path_match": "metrics.*"` targets a whole subtree regardless of the leaf names inside it. Both accept wildcards and both have negating `unmatch` counterparts.

saying these in an interview costs you the question

  • Treats dynamic_templates as an unordered rule set
  • Expects templates to retype fields already in the mapping
  • Confuses match on leaf name with path_match on full path
  • Thinks templates also limit how many fields can be created
  • Assumes templates still fire under dynamic strict

context