skip to content

Why can't you act on a streamed tool call's arguments before the stream ends?

level: seniorimportance: should knowfreq 40%

answer

  1. Fragments, not a document
  2. A prefix is not a prediction
  3. The name comes first, the arguments come last
  4. Parse once, at the completion event
  5. A broken stream means it never happened

basics

~20 s

Arguments arrive as incremental fragments of a JSON string, so the buffer is syntactically invalid until the final chunk lands. Nothing can be parsed, validated or executed from a partial call — only the tool name, which arrives first, is usable early.

solid answer

~50 s

When you stream a response, a tool call does not arrive whole. The provider sends the tool name and call id early, then the arguments object as a sequence of raw JSON string deltas that you concatenate. Mid-stream the buffer is something like `{"store": "brigh` — not a valid document, not parseable, and not safely guessable, because the very next chunk can change what a half-written value means. So you cannot validate against the schema, and you certainly cannot execute: a tool fired on partial arguments may hit the wrong store, the wrong record, or the wrong account. The correct pattern is to accumulate deltas until the call is signalled complete, then parse once and run. What you *can* do early is user-facing: the tool name arrives first, so you can render "checking stock…" immediately, which is where the perceived-latency win actually comes from.

go deeper

for a junior

Know that a streamed tool call arrives in pieces and that you concatenate the argument fragments until the call is complete before parsing or running anything.

for a middle

Explain why a partial arguments buffer is unusable — it is invalid JSON, and even a parseable prefix can change meaning with the next chunk — and note that the tool name arrives early enough to drive the interface.

for a senior

Show the operational rules: parse once at the completion event, never persist a truncated call, roll back on a broken stream, and test with deliberately awkward fragment boundaries rather than fast stubs.

for a principal

Own the boundary between generation and side effect. Decide where a call becomes committed, how cancellation is handled on either side of that line, and how proxy buffering in your delivery path is monitored so the early signal survives.

## How a tool call streams Streaming exists so that output becomes visible before generation finishes. For text that is straightforward — tokens are meaningful as they arrive. For a tool call it is not, because the payload is structured. The provider emits, in order: an event announcing that a tool-call block has started, carrying the tool's name and the call id; then a series of deltas each holding a fragment of the arguments as a raw JSON string; then an event marking the block complete. Your job during the delta phase is concatenation, nothing more. Only after the completion event is the accumulated string a document you can hand to a JSON parser. ## Why partial arguments are worthless, not just risky Three separate problems compound. **Syntactic invalidity.** `{"store": "brigh` is not JSON. Any standard parser rejects it. Reaching for a lenient or streaming JSON parser to "get a preview" trades a clean failure for a dangerous success. **Semantic instability.** Even a fragment that happens to parse can mean something else once complete. `{"limit": 1` becomes `{"limit": 100}`. `{"scope": "one"` becomes `{"scope": "one_region"}`. `{"dry_run": t` looks like it will be `true` and is. A prefix is not a prediction. **Ordering is not stable in a useful way.** Fields do not necessarily arrive in schema order, so "I already have the id, I can start" is a bet on emission order that no contract backs. Together these mean there is no safe early execution — not even for a read. A lookup fired against `brigh` returns nothing or errors, and if your loop feeds that back as a result, the model reasons over a failure you manufactured. ## What you can legitimately do early The name and the id arrive at block start. That is enough for everything in the user interface: show that a named step has begun, start a spinner, display the tool's human-readable label, begin the trace span, start a timer for a timeout you will enforce once the call runs. Users perceive an agent as responsive when they can see *what* it is doing; they do not need to see the arguments being typed. Rendering the raw argument fragments, in fact, tends to look broken — half-written identifiers flickering into different values. ## Cancellation and partial calls Streams break. A client disconnects, a proxy times out, the user hits stop. If the stream ends during the delta phase you hold an incomplete call, and the rule is simple: it never happened. Do not execute it, do not persist it as an assistant turn, and do not fabricate a result for it. An assistant turn recorded with malformed arguments will be replayed on the next request and can confuse or outright break the continuation. Roll back to the last complete turn. The converse also matters: if the stream completes and you *do* execute, cancellation after that point cannot un-run a side effect. The clean boundary is the completion event. ## Proxies and buffering Streaming's benefits are easily erased in transport. A reverse proxy or gateway that buffers the response will hold every delta until the whole body is ready, and the front end sees one large chunk. Symptomatically this looks like streaming that stopped working; the client code is usually fine. It matters here because the early tool-name signal — the one genuinely useful early artifact — is exactly what gets swallowed. ## Testing this properly The partial-argument bugs do not reproduce against fast local stubs, because in tests the deltas often arrive in one batch. Deliberately fragment argument strings at awkward boundaries — inside a string literal, between a key and its colon, mid-number — and assert that your runtime accumulates without parsing and without dispatching. Also test a stream that dies mid-arguments and assert that no tool ran and no assistant turn was persisted. ## The summary rule Accumulate deltas, parse once, then validate, then execute. Use the early name for the interface, never for dispatch. A partially received call is not a call.

  • So what is the actual benefit of streaming a turn that turns out to be a tool call?
    Perceived latency and control, not earlier execution. The tool name and call id arrive at block start, so the interface can immediately show which step began, open a trace span, and start a timeout clock, and the user sees progress instead of a frozen screen. You also get the option to cancel the generation before the call completes, which is impossible with a non-streamed request that returns only when finished.
  • A stream dies halfway through a tool call's arguments. What goes into the conversation history?
    Nothing from that turn. Roll back to the last complete assistant turn and retry from there. Persisting a truncated tool-call block means it gets replayed on the next request with malformed arguments and an unanswered call id, which either errors at the provider or corrupts the model's view of what happened. Treat the block-complete event as the only point at which a call becomes real.
  • Why not use an incremental JSON parser to preview the arguments as they arrive?
    Because a parseable prefix is not a correct value. A partial number, string or enum can change meaning entirely with the next chunk, so anything you build on the preview may be wrong. For rendering a read-only progress hint it is merely cosmetic risk, but for dispatch, validation, or optimistic side effects it converts a clean parse failure into a confident wrong action against real data.

saying these in an interview costs you the question

  • Executing a tool from partially received arguments
  • Assuming argument fields arrive in schema order
  • Persisting a truncated tool call into history
  • Treating a parseable prefix as the final value
  • Blaming the client when a proxy buffers the stream

context