A service mesh is often sold on giving you 'free' observability — uniform metrics, distributed tracing, and access logs for every service without code changes. What exactly does the mesh actually provide here, what's the catch, and what production failure mode commonly trips up teams relying on it?

## Why the mesh is so well placed for this A sidecar mesh sits on every network hop in the system, which puts it in a genuinely privileged position for observability: it sees every request's method, path, response code, and latency, for every service, automatically, the moment sidecar injection is turned on. This is real value: - **no team has to remember** to add a metrics library; - **no language is left uninstrumented**; - and the data is **consistent in shape** across the whole fleet because it comes out of the same proxy implementation. ## The three pillars Concretely, this breaks into three pillars. - **Metrics:** the sidecar emits per-request counters and histograms — request count by response code, request duration percentiles, bytes in/out — labeled by source and destination service, which a metrics backend like Prometheus scrapes and dashboards typically surface as the 'golden signals' (rate, errors, duration) for every service pair in the mesh. - **Access logs:** each sidecar can emit a structured log line per request it handles, giving a request-level audit trail without the app writing anything. - **Distributed tracing:** each sidecar, on seeing a request, generates or forwards trace spans representing this hop, this long, this status, and exports them to a tracing backend. ## The catch: one sidecar, one hop Here is the catch, and it's the single most important nuance in mesh observability: a sidecar only ever sees one hop of a request — it has no inherent knowledge that the request it's proxying from A to B is part of a longer chain that started at a client and continues on to C, D, E. Two separate sidecar-generated spans for A-to-B and B-to-C will only be recognized as part of the same end-to-end trace if they share a common trace ID, and propagating that trace ID across the hop from B's inbound request to B's outbound calls is something only the application code inside B can do — by reading the incoming tracing headers off the request it received and copying them onto any downstream calls it makes. The mesh cannot inject this continuity into a request's business logic layer because it has no visibility into which outbound call, if any, a given inbound request is causally driving; that causal link only exists inside the application. So distributed tracing genuinely is a **mesh-plus-app-code collaboration**: the mesh generates the spans and exports them, but the application has to carry the trace context header through. ## What goes wrong This produces a specific, common production failure mode: a service written by a team unaware of, or that forgot about, this requirement doesn't propagate the tracing headers from its inbound request onto its own outbound calls. The result isn't an error or an alert — everything keeps working, every service's per-hop metrics keep looking healthy, dashboards show normal latency and error rates. What silently breaks is **trace continuity**: instead of one coherent end-to-end trace showing a request's full path through five services, the tracing backend shows two or more disconnected trace fragments with no link between them. An on-call engineer debugging a slow end-to-end request during an incident opens the tracing UI expecting to see the whole path and instead sees the trace mysteriously stop at the service that dropped the headers, making it look like that service is the origin of a new, unrelated request rather than the middle of a chain — actively misleading during exactly the moment an incident demands accurate tracing. ## When the trace you need was never taken A second, related failure mode is sampling misconfiguration: because tracing every single request at high volume is expensive (storage and processing cost), meshes are typically configured with a trace sampling rate well below 100%. During a low-traffic-volume incident affecting a specific customer or a specific rare code path, the odds that the one problematic request was actually sampled and traced can be low, leaving engineers without a trace for the exact request they need, and having to fall back to correlating logs and metrics by timestamp instead — slower and less precise. ## Where it shows up A concrete real-world pattern: Istio's own documentation and observability guides explicitly call out that applications must propagate trace headers for tracing to work correctly across multiple hops, listing this as a required application-side responsibility distinct from anything sidecar injection provides automatically — one of the few pieces of 'you still have to write a little code' guidance in an otherwise code-free mesh adoption story.