How do you trace a LlamaIndex query end-to-end with Arize Phoenix or LlamaTrace?

## Turning it on Observability in LlamaIndex is deliberately a one-liner. `set_global_handler("arize_phoenix")`, imported from `llama_index.core`, installs a callback handler globally; the string names an integration that must be installed as a separate package. Arize Phoenix runs locally — you launch it and open a local UI — while LlamaTrace is its hosted counterpart, reached by passing an endpoint and authenticating through the OTLP headers environment variable that the OpenTelemetry exporter reads. The ordering rule catches people out: install the handler **before** you build indexes, retrievers and query engines. Components capture the callback manager when they are constructed, so a handler installed afterwards produces a suspiciously empty trace view and a wasted afternoon. ## What a trace contains One query becomes a tree of spans, and the tree shape mirrors the pipeline: - The top-level query-engine span, with total latency. - An embedding span for encoding the query. - A retrieve span holding the returned nodes — their text, their metadata and their similarity scores. This is the single most valuable payload in the trace. - A rerank span, if a node postprocessor ran, showing what order changed. - One or more synthesis spans, each with the fully rendered prompt actually sent to the model — after template substitution and context stuffing — and the raw completion, with token counts and latency attached. Seeing the rendered prompt matters more than it sounds. Templates, system prompts, chat history and retrieved context are assembled by layers you did not write, and the difference between what you think you sent and what you sent is where a surprising number of bugs live. ## The debugging algorithm Tracing turns "the answer was wrong" into a decision procedure with two branches: 1. Open the retrieve span and ask whether the fact needed to answer correctly is present in any returned node. If it is **not**, the generator never had a chance — the problem is upstream: chunk size cutting the fact in half, too small a top-k, an embedding model that does not separate your domain vocabulary, or a metadata filter excluding the right document. No amount of prompt work fixes it. 2. If the fact **is** present and the answer still missed it, the problem is downstream: the synthesizer's strategy, prompt wording, context ordering, or the model itself. Look at the rendered prompt — often the fact is buried in position seven of ten chunks, or a refine chain overwrote a correct intermediate answer. This two-branch split is the whole reason to run tracing rather than print statements. It stops the team from tuning prompts to fix a retrieval bug. ## Beyond single-query debugging Spans carry latency, so a trace view also answers the operational question — where did the eight seconds go? Typically it is either a slow rerank model, a synthesizer making several sequential LLM calls, or a cold vector store. The span tree shows the sequential structure directly, including which parts could have run concurrently and did not. Phoenix and LlamaTrace also let you attach evaluation results back onto traced spans, which closes the loop: sample production traffic, score it with the faithfulness and relevancy evaluators, and view failing scores alongside the exact retrieval that produced them. That is materially better than a spreadsheet of scores with no context. ## Alternatives and custom handlers For local, service-free debugging, `LlamaDebugHandler` from `llama_index.core.callbacks` records events in-process; `get_llm_inputs_outputs()` returns the prompt/response pairs, which is enough to answer "what did we actually send" without standing anything up. For production telemetry that must land in your own stack, the `llama_index.core.instrumentation` module is the modern extension point: get the dispatcher, and register a `BaseEventHandler` or `BaseSpanHandler` subclass that forwards events wherever you need. That is how you emit into an existing OpenTelemetry pipeline or your own metrics store rather than a vendor UI. ## Operational cautions Traces contain the retrieved document text and the full prompt, so they contain whatever is in your corpus — treat the trace store as data of the same sensitivity as the source documents, and think about retention and access before pointing production at a hosted endpoint. Full-fidelity tracing of every request is also volume you pay for in storage; sampling, with a rule that always keeps traces for low-scoring or errored requests, is the usual compromise.