A terms aggregation fails with CircuitBreakingException — how do you diagnose and fix it?

## What a circuit breaker is Elasticsearch runs several **circuit breakers**, each of which estimates how much memory an operation is about to need and refuses it with `CircuitBreakingException` if that would push a tracked total past a configured fraction of the JVM heap. The point is failure containment: one bad request returns an error to one caller instead of driving the node into unbounded garbage collection, where it becomes unresponsive, drops out of the cluster, and takes its shards' recovery cost with it. The breakers you meet around aggregations are: - **request** (`indices.breaker.request.limit`, 60% of the heap by default) — per-request data structures, which for aggregations means bucket arrays, ordinal maps for that request, and the sketches behind `cardinality` and `percentiles`. - **fielddata** (`indices.breaker.fielddata.limit`, 40% by default) — uninverted `text` fielddata and global ordinal maps. - **parent** (`indices.breaker.total.limit`) — the sum across breakers plus, when real-memory accounting is enabled, actual heap usage. It trips at a high fraction of the heap and is the one that fires when the node is simply full. - **in_flight_requests** and **accounting** — inbound transport payloads, and memory held by open Lucene segments. ## Reading the failure The exception text names the breaker, the estimated bytes for this request, and the current usage against the limit. That is your first fork: - `[request]` — this aggregation is intrinsically too big. Look at `size` on the `terms` aggregation, the number of nested sub-aggregation levels (buckets multiply), the span and interval of a `date_histogram`, and `precision_threshold` on `cardinality`. - `[fielddata]` — someone is aggregating on a `text` field with `fielddata: true`, or global ordinals for a very high-cardinality `keyword` field have grown large. `GET _cat/fielddata?v&s=size:desc` names the field. - `[parent]` — the request may be innocent. Look at what else is running: `GET _nodes/stats/breaker` for `tripped` counts per breaker, `GET _tasks?actions=*search*&detailed` for concurrent searches, and the search slow log. A single node tripping while its peers are fine usually means a hot shard rather than a global problem. A related but distinct failure is `too_many_buckets_exception`, thrown when a request would produce more buckets than the `search.max_buckets` cluster setting (65,536 by default). That is a bucket-count guard, not a memory breaker, and its fix is the same family of changes. ## Fixing the shape of the work **Shrink the request.** A `terms` aggregation with `"size": 100000` asks every shard for `shard_size` candidate buckets and merges them on the coordinating node. If you genuinely need every bucket, do not use `terms` at all — use the `composite` aggregation, which streams buckets in pages with an `after` key and holds bounded state. **Aggregate on the right field.** Move off `text` and onto a `keyword` sub-field so the values come from doc_values on disk rather than heap fielddata. **Filter before you aggregate.** Most oversized aggregations are oversized because the query matched everything. Narrowing the time range, or routing to the right data tier, removes the work rather than budgeting for it. **Bound the depth.** Nested `terms` inside `terms` multiplies bucket counts. `"collect_mode": "breadth_first"` on the outer aggregation defers sub-aggregation collection until the surviving parent buckets are known, which is the right mode when the outer field is high-cardinality but you only want a few top buckets. **Sample when the answer may be approximate.** A `sampler` aggregation restricts expensive sub-aggregations to the top-scoring documents per shard. **Move the work off the query path.** A continuous transform writing a pre-aggregated summary index, or downsampling for time-series data, turns a nightly heap crisis into a cheap lookup. **Then, and only then, consider capacity.** More heap, more nodes, or smaller shards genuinely help — but heap above roughly 30 GB loses compressed object pointers, so scaling out beats scaling up. ## Why not just raise the limit Because the breaker is not the problem; it is the detector. Raising `indices.breaker.request.limit` lets the same request allocate more, and the outcome is a node in continuous full GC — far worse operationally than an error returned to one client, because it takes every other query on that node with it. If you are convinced the limits are wrong for your workload, change them deliberately as a capacity decision with monitoring, not as a reaction to a single failing query. ## The interview shape Interviewers use this question to separate people who read the error from people who search for a setting to turn up. Say which breaker, say what that breaker protects, name the property of the request that made it large, and reach for the query shape before the configuration.