skip to content

A Grafana dashboard variable named `service` is multi-value with 'Include All' enabled. Explain what Grafana actually substitutes into a panel's query text when the user selects two services or 'All', and how format modifiers such as `${service:regex}`, `${service:pipe}`, `${service:csv}`, `${service:sqlstring}` and `${service:raw}` change that.

level: middleimportance: must knowfreq 52%

answer

  1. substitution into query text, not a bound parameter
  2. multi-value → regex (a|b) / glob {a,b} / quoted CSV
  3. exact-match operator + regex format = empty panel
  4. All expands to every option unless custom all value .*
  5. :raw = no escaping; :sqlstring for SQL

basics

~20 s

Interpolation is textual: Grafana replaces $service in the query string before sending it, and for multi-value it joins the selected values using a format appropriate to the data source (regex alternation, a pipe list, a CSV, quoted SQL strings). All expands to every option unless you set a custom all-value. :raw disables escaping entirely.

solid answer

~50 s

Grafana interpolates **into the query text**, not as a bound parameter. A single value is a plain substitution. A multi-value selection is joined by a *format*, chosen either by the modifier you write or by the data source's default: a metrics store defaults to a regex alternation `(a|b)`, a Graphite-style source to a glob `{a,b}`, SQL sources to a quoted, comma-separated list. Syntax: `$service`, `${service}` (needed when adjacent characters would be swallowed), `${service:format}`. Useful formats: `regex`, `pipe`, `csv`, `json`, `singlequote`/`doublequote`, `sqlstring`, `glob`, `lucene`, `percentencode`, `queryparam` (builds `var-service=a&var-service=b` for links), `text` (the display label rather than the value) and `raw` (no escaping at all). `All` expands to the full option list in that same format — which is expensive and can exceed query length — unless you configure a *custom all value* such as `.*`, which passes one wildcard instead. Because substitution is textual, a variable in a SQL query is an injection sink: use `${var:sqlstring}` and never `:raw` for user-supplied text.

code

text · 9 lines
text
selection = [auth, billing]

$service              -> (auth|billing)      # regex, typical metrics default
${service:pipe}       -> auth|billing
${service:csv}        -> auth,billing
${service:glob}       -> {auth,billing}
${service:sqlstring}  -> 'auth','billing'
${service:queryparam} -> var-service=auth&var-service=billing
${service:raw}        -> auth,billing        # no escaping applied

go deeper

for a junior

Say it is textual substitution, know $var versus ${var}, and know that 'All' means every option unless a custom all value is set.

for a middle

Explain multi-value formatting per data source, name three or four format modifiers with their output shape, and give the exact-match-versus-regex failure.

for a senior

Add the cost model of 'All' expansion, the custom all value trade-off, queryparam for cross-dashboard links, and treat :raw as an injection decision.

for a principal

Rule on it org-wide: mandatory custom all values on high-cardinality variables, sqlstring for SQL sources, raw banned outside reviewed provisioned dashboards, and interpolated-query inspection as part of dashboard review.

## Interpolation is string substitution Before a panel's query leaves Grafana, every `$variable` reference in the query text is replaced with the current selection. There is no parameter binding: the data source receives a finished string. Everything else in this topic follows from that single fact — the escaping problem, the `All` blow-up, and the injection risk. ## Reference syntax - `$service` — the plain form; works when the next character cannot be part of an identifier. - `${service}` — the braced form; required when the reference abuts other text (`${service}_total`), and required to attach a format. - `[[service]]` — a legacy form still parsed but deprecated; do not write new dashboards with it. - `${service:format}` — explicit formatting. Variables interpolate in more than queries: panel titles, text panels, data links, and the queries of *other* variables. ## Single value versus multi-value With multi-value off, the substitution is the selected value verbatim (subject to escaping). With multi-value on, Grafana must turn a *list* into something the target query language understands, and there is no universal answer — hence formats. The data source plugin declares a default: a Prometheus-style source formats as a regex alternation `(auth|billing)` so the query must use a regex-match operator rather than equality; a Graphite-style source formats as a glob `{auth,billing}`; SQL sources format as quoted CSV so the variable belongs inside an `IN (...)`. This is the most common real defect: the panel query uses `label="$service"` (exact match), works fine for one selection, and returns nothing the moment a second value is selected because the substituted text is now `(auth|billing)` compared with `=`. The fix is to write the query for the format you asked for — regex-match operator for regex format. ## The format modifiers - `regex` — escapes each value for regex safety and joins with `|` inside a group. - `pipe` — `a|b` with no grouping. - `csv` — `a,b`. - `json` — a JSON array. - `singlequote` / `doublequote` — quotes each element and joins with commas. - `sqlstring` — single-quotes each element *and* escapes embedded quotes; the correct choice for SQL sources. - `glob` — `{a,b}`. - `lucene` — an OR-joined Lucene clause for search backends. - `percentencode` / `queryparam` — URL-safe; `queryparam` emits `var-service=a&var-service=b`, which is exactly what you need when building a data link to another dashboard that carries the current selection. - `distributed` — repeats the variable name per value, for query languages that need `series, name=a, name=b`. - `text` — the *display label* rather than the value; handy in panel titles when the value is an opaque id. - `raw` — no escaping whatsoever. It exists for the case where the variable holds a fragment of query syntax you deliberately want inlined. ## The `All` option `Include All` adds an `All` entry whose default behaviour is to expand to **every option in the list**, formatted like any other multi-selection. On a variable with 800 pods, that produces an enormous regex or IN-list; you can hit query-length limits, blow up the backend's parse time, or simply make the dashboard slow for no benefit. The remedy is the **custom all value**: set it to `.*` for a regex-matched source or `%` for SQL-style `LIKE`, and Grafana substitutes that single token instead of enumerating. The trade-off is that a wildcard also matches values that were never in the option list — usually what you want, occasionally not (a dashboard scoped by a filtered variable will silently widen). ## Injection Because interpolation is textual, a Text box variable or a value pulled from a query is untrusted input pasted into a query string. For SQL data sources this is straightforward SQL injection, and the mitigation is the same as anywhere: prefer the structured/typed path where the data source offers one, otherwise use `${var:sqlstring}` so quotes are escaped, and treat `:raw` as an explicit decision that this variable is trusted dashboard-author content, never end-user content. Grafana's own permissions model matters here too: a viewer who can edit a panel query on an ad-hoc basis inherits the dashboard's data-source credentials. ## Diagnosing The panel inspector's *Query* tab shows the fully interpolated query that was actually sent. When a panel is empty after a multi-select, read that string first: nine times out of ten you will see `(a|b)` sitting next to an `=` operator, or an `All` expansion that never fit. ## How to answer State 'textual substitution before the query is sent', then multi-value formatting with one concrete mismatch example, then `All` expansion versus custom all value, and close on `:raw` being an escaping opt-out with an injection consequence.

  • A panel works with one service selected and returns no data with two. What is the likely cause?
    The query uses an exact-match operator while the multi-value format produced a regex alternation like `(auth|billing)`. Either switch the query to a regex-match operator or force a different format with a modifier. Confirm by opening the panel inspector's query tab and reading the interpolated string that was actually sent.
  • Why set a custom all value of `.*` instead of letting 'All' expand?
    The default 'All' enumerates every option in the list, so a high-cardinality variable produces a huge regex or IN-list that is slow to parse, may exceed query-length limits, and gains nothing. A single wildcard token is O(1) to substitute and lets the backend match natively. The caveat is that a wildcard also matches values outside the current option list, so a deliberately filtered variable becomes wider than intended.
  • How do you carry the current variable selection into a link to another dashboard?
    Use the `queryparam` format — `${service:queryparam}` renders `var-service=auth&var-service=billing`, the exact URL form Grafana reads back. Appending that to a data link or dashboard link preserves multi-value selections, whereas hand-building `var-service=$service` breaks as soon as more than one value is selected.

saying these in an interview costs you the question

  • Believing Grafana passes variables as bound query parameters — it is textual substitution.
  • Writing an exact-match operator against a multi-value variable and calling it a Grafana bug when the second selection returns nothing.
  • Using `:raw` routinely 'because the other formats break my query', which removes escaping and opens SQL injection.
  • Leaving 'Include All' on a high-cardinality variable with no custom all value.
  • Thinking `$var` and `${var}` differ in meaning rather than only in parsing — and that only the braced form can take a format.

context