Your gen_ai.usage token attributes don't match the provider bill — what do you check?

## Why this question is asked The cost dashboard is the first thing a business stakeholder looks at, and it is almost never exactly right on the first attempt. The span attributes are honest about what the instrumentation saw; the invoice is what the provider charged. The gap between them is a checklist, and knowing the checklist is what separates someone who has run this in production from someone who has read the docs. ## Under-count causes **Streaming without usage.** When a response is streamed, token counts are not implicit in the chunks — the provider reports them only if the client asked for usage to be included. If it did not, the instrumentation has nothing to record and the usage attributes are simply **absent**. Absent is not zero: a `sum()` skips it silently, so a service that streams everything can contribute almost nothing to your token chart while burning real money. This is the single most common cause, and it is worth checking first because it is easy to confirm — look for spans that have `gen_ai.request.model` but no `gen_ai.usage.output_tokens`. **Sampling.** If traces are head-sampled at, say, 10%, your token sum is a tenth of reality, and scaling it back up is only valid if sampling is uniform and independent of request size — which it often is not once someone adds a rule that always samples errors or slow requests. Token totals are one of the strongest arguments for deriving cost from metrics rather than from sampled spans. **Uninstrumented paths.** Calls from a background job, a different language service, a notebook, or a provider whose client the instrumentation does not cover never produce spans at all. The invoice sees them; your trace store does not. **Dropped spans.** Exporter queue overflow under load, or ingestion rejecting oversized spans, removes requests from the aggregate — and because oversized usually means large prompt, the loss is biased toward the expensive calls. ## Over-count causes **Overlapping instrumentation.** If both a framework-level instrumentation and the underlying provider-client instrumentation are active, one logical call can produce a parent and a child span that each carry usage. Summing across all spans then double counts. The fix is to sum at one level — filter by operation or by span depth — and to confirm by opening a single trace and counting how many spans in it carry usage attributes. **Retries and fallbacks.** A retried call is genuinely two provider requests and, if both consumed tokens, genuinely two charges — so this is often not a bug in your data but a bug in your interpretation. Distinguish "requests my application made" from "requests the provider served", and be explicit about which one the dashboard shows. **Evaluation and judge traffic.** LLM-as-judge scoring, synthetic data generation and test suites hit the same API key. If they are instrumented, they inflate your "application" tokens; if they are not, they inflate the invoice relative to your traces. Either way, tag them so they can be split out. ## The pricing layer Even with a perfect token sum, cost can be wrong because tokens are not the whole billing story: - `gen_ai.usage.input_tokens` is one number and does not separate cached from uncached input, which many providers bill at different rates. A prompt-caching rollout can cut the bill sharply with no visible change in the attribute. - Output tokens for reasoning-style models may include tokens you never see in the text, and those are billed. - The backend multiplies by its own price table, keyed by model. If your `gen_ai.response.model` is a snapshot the table does not know, the entry may fall back to a default price or be excluded entirely. - Batch or discounted tiers, and per-contract rates, are invisible to the tracing tool. ## How to actually diagnose it 1. **Reconcile one request, not the aggregate.** Make a single call with known content, find its trace, and compare its attributes against the provider's own usage reporting for that call. Nearly every cause above is visible at n=1. 2. **Count spans per logical call** in that trace to catch duplication. 3. **Query for spans missing usage attributes**, grouped by service and by streaming versus non-streaming. 4. **Check sampling configuration** before trusting any total, and prefer an unsampled metric for cost if the volume justifies it. 5. **Split traffic by tag** — application, evaluation, background jobs — so the comparison against the invoice is like for like. 6. **State the residual error.** "Traces account for 94% of the invoice, the remainder is uninstrumented batch jobs" is a far better answer than a number that claims to be exact. ## The framing to give an interviewer The span is a record of what the instrumentation observed; the invoice is a record of what the provider served. Reconciling them means naming every place those two populations differ — missing spans, missing attributes, duplicated spans, extra provider requests, and billing dimensions the attributes do not carry.