When should you pass disable_batch=True to OpenLLMetry's Traceloop.init(), and what does it cost?

## Two export rhythms, one flag `Traceloop.init()` accepts `disable_batch`, a boolean that is `False` by default. It selects between the two OpenTelemetry export rhythms: batching (accumulate finished spans, ship them periodically or when the buffer fills) and immediate export (ship each span when it ends). OpenLLMetry exposes it as a single flag because in practice the choice follows from one property of your process — how long it lives. ## Why batching is the default A long-running service handles many requests, each producing several spans. Exporting each one individually would mean a network call per span, on a thread the request is waiting on. Batching amortizes that: spans go into a buffer, a background worker ships them in groups, and the request path pays almost nothing. For a service, that is unambiguously the right default. ## Why batching loses data in short-lived processes Buffering only works if someone drains the buffer. A process that exits — or a serverless runtime that *freezes the execution environment* the moment the handler returns — may never reach the point where the pending batch is sent. The spans exist, they were recorded correctly, and they never leave the machine. The symptom is maddeningly specific: the code clearly ran, the model was clearly called, and the backend shows nothing. The situations where this bites are all variations of "the process is about to end": - **Serverless functions.** After the handler returns, the environment can be frozen instantly; background threads do not get scheduled again until the next invocation, if there is one. - **CLI tools and batch jobs.** The script calls the model, prints a result, exits. - **Notebooks.** You run a cell, look for the trace, and it is not there yet. - **Test suites.** Especially ones that assert on emitted telemetry, where determinism matters more than throughput. Setting `disable_batch=True` removes the dependency on a later flush entirely: by the time the span ends, it has already been sent. ## What it costs Immediate export is synchronous work attached to span end. Each finished span becomes its own export call, on the thread that finished it. Three consequences follow: 1. **Added latency per span.** The caller waits for the export, so a slow or unreachable collector shows up as application slowness rather than as missing telemetry. 2. **More network calls.** One per span rather than one per batch, which matters to the collector and to any per-request billing on the receiving end. 3. **No smoothing.** A burst of spans becomes a burst of exports rather than a steady drip. At a few spans per invocation this is invisible. At production request volume it is a self-inflicted performance problem, and it is a genuine interview red flag to propose it as a blanket setting "so we never lose traces". ## Choosing correctly The decision rule is short: **if the process outlives many requests, batch; if the process is about to exit, do not.** In a container running a web server, leave the default. In a Lambda, a Cloud Run job, a Celery one-shot, a script or a notebook, set `disable_batch=True`. A useful nuance for interviews: a long-running service still loses its final buffer on an abrupt kill. Graceful shutdown — letting the process run its shutdown path rather than sending an immediate hard kill — is what preserves those last spans, not switching the whole service to immediate export. If your orchestrator's termination grace period is too short, you lose the tail of your telemetry during every deploy, and the fix is the grace period, not the flag. ## Where the boundary is `disable_batch` is the OpenLLMetry-level convenience for this choice. The detailed tuning knobs of the underlying batching processor — queue size, schedule delay, maximum batch size, export timeout — belong to OpenTelemetry SDK configuration rather than to OpenLLMetry, and you reach them by supplying your own processor rather than through this flag. Knowing where that line sits is itself a good signal in an interview: OpenLLMetry chooses sane defaults and exposes the one decision that most applications actually need to make.