How do tags and metadata make LangSmith runs findable, and where do you set them?

## Why this is the difference between a trace store and a haystack A production LangSmith project accumulates thousands of runs an hour. The debugging questions you actually get are not "show me the last run" — they are "show me what happened for the customer who complained at 14:20", "compare the canary prompt against the current one", "find every run of the summariser that errored since the deploy". None of those are answerable from run names and timestamps. They are answerable from the labels you put on runs at the time they were created, which means the decision has to be made before the incident, not during it. ## Tags Tags are a flat list of strings on a run. Treat them as **low-cardinality dimensions** — values from a small closed set that you will want to slice by repeatedly: - environment: `"prod"`, `"staging"` - rollout: `"canary"`, `"baseline"` - experiment arm: `"variant-a"`, `"variant-b"` - surface: `"mobile"`, `"web"` A tag is a yes/no membership test. In the filter language that is `has(tags, "canary")`, composable with `and(...)` and `or(...)`. Do not put a user id or a request id in tags: you end up with an unbounded tag vocabulary that is useless as a facet. ## Metadata Metadata is a JSON-serialisable dict on the run, and this is where identifiers and versions belong: - `user_id`, `tenant_id`, `request_id` - `prompt_version`, `model_revision`, `git_sha` - feature-flag state, retrieval index name You filter on key/value pairs, so `prompt_version: "v7"` gives you every run of a specific prompt release, and `git_sha` lets you attribute a regression to a deployment. Anything a support ticket might quote at you belongs in metadata — that is the field you will search when someone says "my request failed". ## The three places to set them **On the decorator** — for facts that are true of every execution of that function: ``` @traceable(tags=["retrieval"], metadata={"index": "docs-v3"}) ``` **Per call** — pass `langsmith_extra` as a keyword argument to the decorated function; the decorator consumes it and your function never sees it. Useful when the caller knows something the callee does not. **Per request scope** — `with tracing_context(tags=[...], metadata={...}):` wraps a block, and everything traced inside it carries the values. This is the right hook for a web framework: in middleware, open a `tracing_context` with the authenticated user, the request id and the release sha, then call the handler. Every run in the resulting trace is labelled without any of your business functions knowing that LangSmith exists. `tracing_context` also takes `project_name` and `enabled`, so the same hook can route or suppress traffic. ## Searching afterwards The project view exposes a filter box using the same query language you pass to the SDK: `has(tags, "canary")`, equality on run fields, `and(...)`/`or(...)` composition, and time bounds. Programmatically: ``` Client().list_runs(project_name="checkout-assistant", filter='has(tags, "canary")', is_root=True) ``` `is_root=True` restricts to whole-request runs rather than every nested step, which is usually what you want when counting or exporting. `run_type=` narrows to one class of step. The results are what you triage, and — separately — what a team later promotes into datasets, which is the practical reason people care about labelling quality: unlabelled traffic cannot be selected on any axis worth selecting on. ## Threads Multi-turn applications need traces grouped by conversation, not just by request. LangSmith reads the metadata keys `session_id`, `thread_id` or `conversation_id` and groups traces carrying the same value into a thread view, so you can read a conversation end to end. It costs one metadata entry set in your request-scope context, and retrofitting it after the fact is impossible for traffic already recorded. ## Anti-patterns Encoding facts into the run **name** (`answer_question_user_1234`) — names are for identifying the step, and a per-user name makes aggregation impossible. Dumping the entire request object into metadata, which bloats every run and can smuggle personal data into the trace store. Using tags for identifiers, producing thousands of one-off tags. And labelling only the root run when the interesting failure is three levels down — a request-scope `tracing_context` avoids exactly that.