skip to content

Which Prometheus metric and label naming conventions do you enforce in review, and what breaks later when each is ignored?

level: seniorimportance: should knowfreq 41%

answer

  1. Names carry the unit, labels carry dimensions
  2. Base units only: seconds, bytes, ratios
  3. Cumulative counters end one particular way
  4. Two leading underscores mean hands off

basics

~20 s

Enforce a namespace prefix, base units in the name (seconds, bytes, 0-1 ratios), a plural unit suffix, _total on a cumulative counter, and dimensions as labels, never name fragments. Label names beginning with two underscores are reserved.

solid answer

~50 s

A metric name is only the value of the `__name__` label, so nothing in Prometheus enforces units, types or naming — review is the enforcement. The conventions: a namespace prefix identifying the application (`seedorder_`); base units spelled in the plural, seconds and bytes, never milliseconds or kilobytes, with ratios as unitless 0-1 values suffixed `_ratio`; `_total` on a cumulative counter, after the unit as in `process_cpu_seconds_total`; every dimension in a label rather than baked into the name; and the reserved namespaces left alone — label names starting with two underscores are stripped before storage, `le` and `quantile` are reserved on histogram and summary series, and a colon belongs to recording-rule output. The cost lands later: mismatched units cannot be compared in one expression, and a dimension in a name means every query is edited when a new value appears.

code

text · 9 lines
text
# before: wrong unit in the name, dimension baked in, counter unmarked
seedorder_checkout_latency_ms_eu_west 187
seedorder_cache_hitrate_pct 91.37
seedorder_orders_eu_west 41823

# after: base units, dimension as a label, _total on the counter
seedorder_checkout_duration_seconds{region="eu-west-3"} 0.187
seedorder_cache_hit_ratio{region="eu-west-3"} 0.9137
seedorder_orders_total{region="eu-west-3"} 41823

go deeper

for a junior

Recall that Prometheus metric names use base units — seconds and bytes — and that a cumulative counter's name ends in _total. Knowing that a region or customer belongs in a label rather than the name is already the main point.

for a middle

Explain where the unit suffix sits relative to _total, why ratios are unitless 0-1 values rather than percentages, and that label names beginning with two underscores are reserved and stripped before storage.

for a senior

Show the downstream cost of each violation: units that cannot be compared in one expression, percentages that cannot be re-aggregated, and names carrying dimensions that force an edit to every query when a new value appears.

for a principal

Own the convention across teams: how it is checked before an exporter ships, what the migration looks like when history already carries the old name, and why that asymmetry makes review the cheapest place to enforce it.

## Why naming is load-bearing in Prometheus A metric name is just the value of the `__name__` label. There is no schema registry, no unit metadata that queries respect, and no type enforcement — the `# TYPE` line is advisory metadata that never reaches a sample. Everything that makes two services' metrics comparable is convention, held up by review. That is what makes naming one of the few Prometheus subjects where a five-minute review comment changes how operable a system is a year later. ## The conventions worth enforcing 1. **A namespace prefix.** A single word identifying the application or subsystem: `seedorder_`, `process_`, `go_`. It prevents collisions between unrelated exporters scraped into the same server and makes the origin of a series obvious at a glance. 2. **Base units in the name, spelled in the plural.** Seconds and bytes — never milliseconds, microseconds, kilobytes or megabytes. A ratio is a unitless value between 0 and 1 with a `_ratio` suffix, never a percentage. 3. **`_total` on a cumulative counter, after the unit.** `seedorder_bytes_written_total`, `process_cpu_seconds_total`. The unit says what is accumulating, `_total` says that it accumulates; reversing them reads as a count of totals. 4. **Dimensions live in labels, never in the name.** `seedorder_orders_total{region="eu-west-3"}`, not `seedorder_orders_eu_west_3_total`. 5. **Leave the reserved namespaces alone.** Label names beginning with two underscores belong to the server. `le` and `quantile` are reserved on histogram and summary series. A colon in a metric name is reserved for the output of recording rules and should never appear in an exported name. ## What each one costs when it is ignored | Convention broken | What breaks, and when | |---|---| | a unit that is not a base unit | one service reports seconds and another milliseconds; no single expression compares or aggregates them, and every panel needs a per-service multiplier | | a percentage instead of a 0-1 ratio | the numerator and denominator are gone, so the value cannot be re-aggregated across instances — averaging percentages over different denominators is simply wrong | | a counter without `_total` | every reader, dashboard author and linter has to guess whether the series is cumulative, and the declared type is metadata that nothing checks | | a dimension baked into the name | a new region means a new metric name, so every query, alert and panel needs editing and nothing can aggregate across the dimension | | a label name beginning with `__` | the label is stripped before the sample is stored, so the dimension you thought you added is silently missing | | a colon in an exported name | it collides with the namespace reserved for recording-rule output | ## The reserved label namespace The double-underscore prefix is not stylistic. `__name__` is the label that holds the metric name, which is why `up` and `{__name__="up"}` select exactly the same series. The rest of that space is used by the server for target and discovery information while it decides what to scrape and how to label it, and labels whose names begin with two underscores are removed before the sample is stored. The rule for anyone writing an exporter is short: you do not own that namespace, and anything you put in it disappears. ## Applying it in review 1. **Read the name aloud without its labels.** Does it say what is measured and in what unit? `seedorder_pack_duration_seconds` does. `seedorder_pack_time` does not. 2. **Ask whether any part of the name is a value rather than a concept.** If it names a region, a customer, a route or a version, it is a label. 3. **Check the suffix against the type.** Cumulative gets `_total`. A histogram family gets `_bucket`, `_sum` and `_count` from the client library, and its base name should still carry the unit. 4. **Check the unit against the code that produces it.** Code that measures in milliseconds and divides before exporting is correct; code that exports milliseconds under a `_seconds` name is a bug that survives for years because nothing in the server contradicts it. 5. **Check for reserved names.** No label name starting with two underscores, and nothing reusing `le` or `quantile` for its own purpose. ## The honest caveat None of this is enforced, and renaming afterwards is not free. Old series keep their old names for the whole retention period, so for a while both names exist and every query, alert and dashboard has to handle the overlap. That asymmetry is the argument for spending the effort at the point the exporter is written: it is nearly free to name a brand-new region's services correctly before they have exported a single sample, and it is a multi-week migration to rename them once a year of history carries the old name.

  • Where does the unit suffix go on a Prometheus counter that accumulates seconds?
    Before the `_total` suffix, as in `process_cpu_seconds_total`. The unit describes what is being accumulated and `_total` says that it accumulates, so the two read naturally in that order. Reversing them into something like `..._total_seconds` reads as a count of totals measured in seconds, which is not what the series holds.
  • What is `__name__` in Prometheus, and why does it matter to naming?
    It is the reserved label that holds the metric name. A series' identity is its full label set including `__name__`, which is why `up` and `{__name__="up"}` select the same series. It also explains why the name has no special powers: it carries no unit, no type and no schema, so every convention about names is convention rather than validation.
  • Why is a 0-1 ratio preferred to a percentage in a Prometheus metric name?
    A percentage is a number the exporter has already divided, which discards the numerator and denominator. That makes it impossible to re-aggregate honestly across instances, because averaging percentages computed over different denominators is arithmetically wrong. Exporting the two underlying counters and letting a query do the division keeps both facts and lets any grouping be computed correctly.

saying these in an interview costs you the question

  • Exports latency in milliseconds because the code uses millis
  • Bakes the region or customer into the metric name
  • Writes a cumulative counter without the _total suffix
  • Tries to set a label name beginning with two underscores
  • Stores a percentage where a 0-1 ratio is conventional
  • Assumes the TYPE declaration enforces the naming