How do you set a prompt and completion payload capture policy for LLM traces?

## Why this needs a policy at all On a conventional service, span attributes are small and bounded. On an LLM span they are not: an assembled prompt can be tens or hundreds of kilobytes, one turn can hold several of them, and the text is exactly the material most likely to contain customer data. Left to each team, the outcome is bimodal — either nobody captures payloads and every investigation dead-ends at 'we cannot see what the model was told', or everybody captures everything and the observability bill and the risk surface both arrive as a surprise. Deciding it once, centrally, with an explicit rationale, is the principal-level move. ## What attribute-only spans already give you Before reaching for payloads, notice how much is answerable without them. Model and operation, latency per span, the finish reason, the number of agent iterations, retrieval candidate counts in and out, tool argument *shapes*, error statuses, span structure. That set answers latency regressions, loop and step-count problems, truncation, and most retrieval-shape bugs. Payloads are needed for one class specifically: 'the structure looks right and the answer is still wrong'. Sizing your capture to that class rather than to everything is the whole trick. ## The levers **Capture rate.** A small sampled percentage of traces gets full payloads. Sample the decision **at the root**, so a chosen trace is captured whole; sampling per span produces traces with payloads on some spans and holes on others, which is worse than none because the missing span is always the interesting one. Stratify by feature and tenant, because a flat percentage of skewed traffic yields nothing at all for the low-volume path that is failing. **Always-on exceptions.** Errors, and turns the application itself marked as suspect, should be captured at 100%. These are rare by definition, so they cost little, and they are the traces someone will actually open. **Size bounds.** Cap payload size, but pick the truncation shape deliberately: a plain prefix cut loses the tail, which is where the answer is, and it silently discards exactly the huge contexts you most wanted to inspect. Record the original length and a stable hash so you can tell a truncated capture from a short prompt, and prefer head-plus-tail truncation, or store-by-reference where the span holds a pointer into object storage and the large body lives outside the trace index. **Redaction on write.** Run payloads through a redaction step in the export path so raw text is never persisted. The detail of what constitutes sensitive content is a separate discipline; the tracing decision is that the hook exists, sits before the write, and fails closed. **Retention split.** Attribute-only spans can live for months cheaply; payloads should not. Different retention for the two is usually the single largest cost lever, and it also shrinks the blast radius of the store. **Tenant controls.** Some contracts forbid capturing content at all. That has to be expressible per tenant, and the default for a tenant with no explicit setting should be the conservative one. ## The tradeoff you should state plainly There is no setting that is right for everyone, and the honest framing is a three-way tension between debuggability, cost and exposure. A team shipping a new agent into an unfamiliar domain should capture aggressively and cut retention hard, because they do not yet know what they will need to look at. A mature, high-volume, regulated product should invert that: rich attributes everywhere, payloads on a thin sample, tight redaction, short retention. Both are defensible. What is not defensible is arriving at either by accident. ## Second-order effects worth anticipating Capture policy quietly determines what investigations are possible months later, so it deserves a review cadence rather than being set once. It also interacts with reproduction: if a failing turn was not among the captured sample, your fallback is to re-run the path with capture forced on, which requires that forcing capture per request is a supported operation rather than a redeploy. Build that switch early — it converts 'we have no data on this incident' into a ten-minute detour. ## Interview shape A strong answer names the levers, gives one concrete default (a low single-digit capture percentage, 100% on errors, payloads retained for days rather than months), explains the root-level sampling requirement, and is explicit that the right point on the curve depends on domain maturity and regulatory exposure. A weak answer is either 'capture everything, storage is cheap' or 'never store prompts' — both skip the actual decision.