skip to content

What is an ObservationConvention, and what is the difference between low-cardinality and high-cardinality KeyValues?

level: seniorimportance: should knowfreq 35%

answer

  1. Convention = name + contextualName + tags
  2. low card = bounded -> metric dimensions (time series)
  3. high card = unbounded (ids) -> spans only
  4. cardinality explosion = high-card value used as metric tag
  5. override DefaultServerRequestObservationConvention

basics

~20 s

An ObservationConvention centralizes how an observation is named and tagged. It produces low-cardinality KeyValues (few distinct values — safe as metric tags/dimensions) and high-cardinality KeyValues (many distinct values like an id — attached to spans only, never to metric dimensions).

solid answer

~40 s

ObservationConvention<T extends Observation.Context> defines an observation's identity: getName(), getContextualName(context), getLowCardinalityKeyValues(context), getHighCardinalityKeyValues(context), and supportsContext(context). It decouples the instrumentation point from tag naming so you can override naming without touching the instrumented code. The crucial distinction is cardinality. Low-cardinality KeyValues have a small, bounded set of values (HTTP method, status class, outcome) and become metric tags/dimensions — every distinct combination creates a separate time series, so low cardinality keeps that manageable. High-cardinality KeyValues have effectively unbounded values (user id, request URL, order id); they'd explode metric storage, so Micrometer attaches them only to spans/traces, not to meters. Frameworks ship default conventions (e.g. DefaultServerRequestObservationConvention); you provide a GlobalObservationConvention or a per-observation convention to customize. The wrong cardinality choice is the classic production incident.

code

java · 30 lines
java
import io.micrometer.common.KeyValue;
import io.micrometer.common.KeyValues;
import io.micrometer.observation.Observation;
import io.micrometer.observation.ObservationConvention;

class UserLookupContext extends Observation.Context {
    final String userId;   // unbounded -> HIGH cardinality
    final String tenant;   // bounded    -> LOW cardinality
    UserLookupContext(String userId, String tenant) { this.userId = userId; this.tenant = tenant; }
}

class UserLookupConvention implements ObservationConvention<UserLookupContext> {
    @Override public String getName() { return "user.lookup"; }

    @Override public String getContextualName(UserLookupContext c) {
        return "lookup user " + c.userId; // span name, per-request detail is fine
    }

    @Override public KeyValues getLowCardinalityKeyValues(UserLookupContext c) {
        return KeyValues.of(KeyValue.of("tenant", c.tenant)); // safe as a metric dimension
    }

    @Override public KeyValues getHighCardinalityKeyValues(UserLookupContext c) {
        return KeyValues.of(KeyValue.of("user.id", c.userId)); // span-only, never a meter tag
    }

    @Override public boolean supportsContext(Observation.Context context) {
        return context instanceof UserLookupContext;
    }
}

go deeper

for a junior

Know a convention names the observation and that some tags are metric-safe and some aren't.

for a middle

Define low vs high cardinality by distinct-value count and know low-card -> metrics, high-card -> spans.

for a senior

Subclass the framework default convention to add bounded dimensions; articulate the cardinality-explosion risk and templated-URI rationale.

for a principal

Own tagging standards and cardinality budgets across services; decide global vs per-observation conventions and their cost implications.

## What a convention is An **`ObservationConvention`** answers: *what is this observation called and how is it tagged?* It centralizes naming/tagging so the code that *starts* an observation doesn't hardcode strings. Interface: ```java interface ObservationConvention<T extends Observation.Context> { boolean supportsContext(Observation.Context context); default String getName() { ... } default String getContextualName(T context) { return null; } default KeyValues getLowCardinalityKeyValues(T context) { return KeyValues.empty(); } default KeyValues getHighCardinalityKeyValues(T context) { return KeyValues.empty(); } } ``` - **getName()** — the base observation/metric name (e.g. `http.server.requests`). - **getContextualName(context)** — a per-instance, human-friendly name used for the **span** (e.g. `http get /users/{id}`). - **getLowCardinalityKeyValues / getHighCardinalityKeyValues** — the tags (`KeyValues` = an immutable set of `KeyValue` name/value pairs). - **supportsContext** — which context types this convention applies to. `GlobalObservationConvention` is a marker subtype that Micrometer applies to *all* matching observations globally; you register it on the registry. You can also pass a convention directly when creating an observation, and framework instrumentation exposes a default convention you can subclass/replace. ## Low vs high cardinality — the core concept **Cardinality** = the number of *distinct values* a tag can take. - **Low-cardinality KeyValues**: a small, bounded value set. Examples: HTTP method (`GET`/`POST`/...), status outcome (`SUCCESS`/`CLIENT_ERROR`/`SERVER_ERROR`), a templated URI (`/users/{id}`, not the concrete id). These become **metric tags (dimensions)**. Each unique combination of tag values is a separate **time series** in the metrics backend. Because the value set is small, the number of series stays bounded and cheap. - **High-cardinality KeyValues**: effectively unbounded distinct values. Examples: user id, concrete request path with the id substituted, order id, SQL bind values. If these became metric dimensions you'd get a near-infinite number of time series — a **cardinality explosion** that can OOM or bankrupt a metrics backend (Prometheus, etc.). So Micrometer attaches high-cardinality KeyValues to **spans/traces only** (where per-request detail is exactly what you want and storage is per-span, not per-series), and **never** to meters. This is the whole reason the API splits the two methods: the *same* observation feeds *both* metrics and tracing, and each sink gets the appropriate subset. ## Example: HTTP Spring MVC's `DefaultServerRequestObservationContext` + convention emit low-cardinality tags like `method`, `status`, `outcome`, `uri` (templated — `/users/{id}`), and can carry high-cardinality detail like the full path on the span. Note the **templated** URI is deliberately low cardinality; the raw URL with ids is not. ## Customizing Subclass the framework default and override just the tag methods, then register your convention as a bean: ```java class MyServerObsConvention extends DefaultServerRequestObservationConvention { @Override public KeyValues getLowCardinalityKeyValues(ServerRequestObservationContext c) { return super.getLowCardinalityKeyValues(c).and("tenant", tenantOf(c)); } } ``` Spring Boot picks up a single bean of the framework convention type for that instrumentation. Prefer overriding the framework convention to renaming tags after the fact. ## Gotchas - **Putting a high-cardinality value in low-cardinality KeyValues** is the classic outage: metric series blow up. Rule of thumb: if you can't bound the distinct values, it's high cardinality. - Tag **keys** should be stable; only some backends tolerate varying key sets per series. - A convention passed explicitly to an observation overrides the global default for that observation. - Conventions decide naming; they don't drop observations (that's a predicate) or record signals (that's a handler). ## When to use Use a custom `ObservationConvention` to standardize names/tags across an org, add a bounded dimension (tenant, region), or reshape framework defaults — while keeping unbounded ids on the span via high-cardinality KeyValues.

  • Why does Micrometer keep high-cardinality KeyValues off metrics but put them on spans?
    Each distinct metric tag-value combination is a separate time series; unbounded values (ids) would explode series count and storage/cost. Spans store per-request detail individually, so high-cardinality data is both safe and valuable there.
  • Why is the templated URI (/users/{id}) used as a low-cardinality tag instead of the raw path?
    The template collapses all concrete ids into one bounded value, keeping the number of time series small; the raw path with ids would be effectively unbounded and cause a cardinality explosion.

saying these in an interview costs you the question

  • Putting a user id / raw URL in low-cardinality KeyValues (cardinality explosion)
  • Thinking low vs high cardinality is about tag length or importance rather than number of distinct values
  • Believing conventions can drop observations (that's predicates) or emit signals (that's handlers)
  • Assuming high-cardinality KeyValues show up as metric dimensions

context

resolving links…