How would you structure Elasticsearch component and index templates for many teams and index families?

## What actually goes wrong at scale A single team with three index families needs no governance. The failure modes appear at a dozen teams and a hundred families, and they are consistent: - Two teams pick the same priority for overlapping patterns and the second `PUT` fails, or worse, they pick patterns that overlap a vendor template and their configuration silently never applies. - A shared component template gains a field for one team and it lands in everyone's indices. - Nobody can answer "why does this index have five shards?" because it takes reading six component templates and knowing merge order. - A mapping fix is deployed and takes effect three weeks later when a family finally rolls over, in an order nobody tracked. Governance is the set of conventions that make each of these either impossible or loud. ## Layer by ownership, not by convenience Give component templates semantic roles with a single owner each. A workable default: 1. **platform-defaults** — cluster-wide index settings the platform team owns: replica count, refresh interval, the ILM policy reference. Nobody else edits it. 2. **schema-common** — the field schema every document shares (`@timestamp`, service and environment identifiers, correlation ids). Changes here are cross-cutting and reviewed as such. 3. **\<domain\>-fields** — one per team or bounded context, owned by that team, holding only its own fields. 4. **\<family\>-overrides** — a reserved, highest-precedence slot per family, usually empty, that exists so an incident fix has somewhere to go without editing shared layers. Each family then gets exactly one index template with narrow `index_patterns` and `composed_of: [platform-defaults, schema-common, domain-fields, family-overrides]`. Precedence is positional, so this list reads exactly as the precedence order, which is the property that makes it reviewable. From 8.7 the override slot can be referenced with `ignore_missing_component_templates` so the index template stays valid while the component does not exist — the same hook Elastic's own integrations leave for user customisation. ## Publish the priority bands `priority` is a bare integer with no built-in meaning, so it needs an external convention. Reserve ranges and write them down: - 0–99: nothing (leave room below the vendor templates rather than fighting them) - 100–199: vendor and integration-managed templates — do not occupy - 200–299: platform baseline templates covering broad families - 300–499: team family templates - 500+: deliberate overrides, including incident response The absolute numbers matter less than that they are documented, that per-family templates sit above per-family-group ones, and that you have checked the priorities of any shipped templates covering your patterns with `GET _index_template`. Ban catch-all `*` patterns outright: a high-priority `*` template claims every index in the cluster including ones you have never heard of. ## Treat templates as code with a real gate Every template is a JSON file in a repository, applied by a pipeline, never by hand in a console. Populate `_meta` with owner, ticket and source path, and bump `version` on each change, so an operator staring at `GET _index_template` can attribute anything they find. The gate that earns its keep is the composed-output diff. In CI, apply the templates to a disposable cluster, run `POST _index_template/_simulate_index/<name>` for every index name pattern the system genuinely creates, and diff the resulting settings and mappings against checked-in golden files. This catches the whole silent class: a dropped field, a `keyword` flipped to `text`, a component reorder, a new integration template that now outranks yours. It also makes review honest, because the artefact reviewers read is the effective mapping rather than a fragment whose merged meaning nobody can compute by eye. ## Plan the latency, not just the change The hardest part to explain to stakeholders is that a merged template change reaches no data at all until an index is created. Every rollout therefore has a second half: which families roll over, when, and which need a reindex because they are not rolling. Reindexing a large family is real I/O and real cluster capacity, so the rollout is scheduled work. For additive, non-breaking field changes you can simply wait for natural rollover; for a type correction, the family must be reindexed behind an alias or accept a mixed mapping across generations — and mixed mappings across an aliased set are their own operational hazard when queries span both. ## Guard the blast radius Two limits belong in the design. Keep an eye on total field count per index — a shared schema component that grows unchecked pushes families toward the field limit and, more importantly, toward slow, memory-hungry mappings. And keep the number of component layers small: every layer added is a permanent tax on every future debugging session. When someone proposes a fifth layer, the right question is whether the concern is genuinely orthogonal to the existing four or is simply a team-shaped fragment that belongs in that team's own component.