skip to content

In OpenTelemetry, when should something be recorded as a span event, as a span link, or as a separate child span? Give the rule for each and an example.

level: middleimportance: must knowfreq 42%

answer

  1. Duration + independent timing → child span
  2. Point in time inside a span → event
  3. Association without parenthood → link
  4. Batch consumer: one span, many links, not many parents
  5. Exception event does NOT set span status

basics

~20 s

Child span: work with its own duration you want timed and nested. Span event: a point-in-time annotation inside a span, with no duration. Span link: an association to a span that is not your parent — commonly a different trace, as in batch processing.

solid answer

~50 s

Ask two questions: does it have a duration, and is it causally *my parent*? Something with a meaningful start and end that you want to time, sample and search independently is a **child span** — a database call, a downstream request, an expensive computation. Something instantaneous that only makes sense inside its parent's lifetime is a **span event**: a name, a timestamp and attributes, with the canonical case being a recorded exception (`exception.type`, `exception.message`, `exception.stacktrace`). Something that relates your span to another span you did not descend from is a **link**: it carries the other span's context plus attributes and asserts association without parent-child causality or nesting. The classic case is a consumer that processes a batch of one hundred messages from ninety different traces — it cannot have ninety parents, so it starts its own span and links to all of them. Cost matters too: spans multiply span count and are individually sampled, whereas events ride along with their parent and disappear if the parent is dropped.

code

text · 6 lines
text
trace X ── producer span p1 ─┐
trace Y ── producer span p2 ─┤   consumer starts a NEW trace Z
trace Z' ─ producer span p3 ─┘   span "process-batch" (kind=CONSUMER)
                                   links: [p1.ctx, p2.ctx, p3.ctx]
                                   events: [{"retry", t=+40ms}]
                                   child spans: db.write (has duration)

go deeper

for a junior

Get the three-way rule right — duration means span, moment means event, other-than-parent means link — with one example each.

for a middle

Add the exception-event convention and the reason a batch consumer must use links rather than a chosen parent.

for a senior

Discuss cost and sampling consequences, event-versus-log-record trade-offs, and uneven backend rendering of links.

for a principal

Set instrumentation policy: what deserves a span at all, where per-item detail belongs (aggregate attributes, metrics), and how sampling changes which of these signals can be relied on for post-incident reconstruction.

## The decision, stated once - **Has its own duration and deserves independent timing/searching → child span.** - **Is a moment in time inside a span's life → event.** - **Relates this span to another span that is not its parent → link.** Everything below is why. ## Child spans A child span is a full span: its own id, its own start and end, its own attributes, events, status. It costs a record in the backend, it participates in the trace tree, and it is the only one of the three that can be timed on its own. Use it for outbound calls, database queries, queue publishes, and internal work whose latency you want to attribute. The failure mode is over-instrumentation: a span per loop iteration or per tiny helper produces traces of thousands of spans that are expensive to store and unreadable to a human. When you find yourself wanting a span for something instantaneous or trivially fast, you probably want an event or an attribute. ## Events An event is `{ name, timestamp, attributes }` attached to a span. It has no duration and no identity, and it cannot be found without finding its parent span first. It is the right shape for: an exception being recorded, a cache hit or miss, a retry attempt starting, a lock being acquired, a state transition. The most important standard case is the exception event, whose conventional name is `exception` with attributes `exception.type`, `exception.message`, `exception.stacktrace` and `exception.escaped`. Note the separation that catches people out: **recording the exception event does not set the span's status**. Handling an exception and continuing is an event on a span whose status stays UNSET; a genuine failure needs the status set to ERROR as a separate act. Events are cheap relative to spans but not free — they are bounded by per-span limits, and once exceeded the model records a `dropped_events_count` rather than growing without bound. Their real weakness is queryability: backends generally index spans far better than events, so something you will want to search across all traces is often better as a span attribute or a log record. ## Links A link holds another span's context (trace id, span id, trace flags, tracestate) plus its own attributes. It says "this span is related to that one" and deliberately says nothing about causality direction, nesting or timing. A span may have many links; they are set at span creation in most SDKs. Why the model needs them: parenthood is single-valued and implies containment. Some real relationships are neither. Three concrete cases: - **Batch consumption.** A consumer pulls a hundred messages produced by many different requests. Parenting the processing span to one of them would be arbitrary and would silently attach ninety-nine unrelated traces' worth of meaning to one. Instead the consumer starts a new trace and links to each producer's span context. - **Fan-in / join.** Several independent operations converge; the joining span links to all contributors. - **Deferred or replayed work.** Something reprocessed hours later from a dead-letter queue should not be nested inside a trace that ended long ago — a new trace with a link preserves the association without producing a trace with an eight-hour root span. The practical caveat is that backend support for links is uneven: many UIs render them as a secondary "related traces" affordance rather than in the waterfall, so do not rely on links being as discoverable as parenthood. ## Events versus log records There is a genuine overlap: an OpenTelemetry log record correlated to the active span carries roughly the same information as a span event — a timestamp, a body/name, attributes, and trace context. The differences in practice are that log records are stored and indexed as their own signal (searchable without the trace, retained on their own schedule, subject to their own volume controls), whereas events are inseparable from their span and vanish with it. If a 1% trace sampler drops the span, its events are gone; a log record survives. The specification has been converging these — event semantics are increasingly expressed on top of the log record schema — so treat "which one" as partly a question of what your backend indexes well. ## Sampling interaction One last consequence worth voicing in an interview: child spans are sampled with their trace, so nothing about them survives a dropped trace either; events likewise. Links are just fields, so a linked-to trace being dropped leaves you holding a reference to a trace that no longer exists. None of this is a defect, but it explains why critical, must-not-lose information belongs in metrics or logs rather than exclusively in trace-attached data.

  • Why can't a batch-processing span simply have multiple parents?
    The data model gives a span exactly one `parent_span_id`, and parenthood implies containment within the parent's lifetime and membership of the parent's trace. A hundred messages from a hundred traces satisfy neither. Links exist precisely to express many-to-one association without claiming containment or forcing all those traces into one.
  • When would you prefer a correlated log record over a span event for the same information?
    When you need it to be independently searchable, retained on a different schedule, or to survive trace sampling. Events are inseparable from their span: if the trace is dropped by a sampler, the event is gone. A log record carrying trace and span ids is stored and indexed as its own signal, so you can query it across all requests and still jump to the trace when one exists.
  • What is the cost of instrumenting a tight loop with one span per iteration?
    Span count multiplies, which inflates export volume, backend storage and cost, and produces traces so large that a human cannot read the waterfall — some backends also truncate very large traces. If per-iteration visibility matters, prefer aggregate attributes on the enclosing span (counts, totals, max), events for the notable iterations, or a metric with a histogram of iteration durations.

saying these in an interview costs you the question

  • Using links where a straightforward parent-child relationship exists
  • Parenting a batch-processing span to one arbitrary message's trace
  • Assuming a recorded exception event sets the span status to ERROR
  • Expecting events to be searchable across traces the way span attributes are
  • Creating a child span for something with no meaningful duration

context