How do you reconstruct tool_calls from an OpenAI streaming chat completion?

## Why streaming complicates tool calls Streaming text is easy: append `delta.content` to a buffer and render. Tool calls are structured, and the API streams them the same token-by-token way — which means you receive a JSON object in pieces, out of a shape you can validate, until the very end. Any code that assumes a whole `tool_calls` array will crash the first time someone flips `stream=True`. ## The delta shape Each streamed chunk has `choices[0].delta`. During a tool call, `delta.content` is null and `delta.tool_calls` is a list of partial entries. A partial entry has: - **`index`** — the position of this call within the eventual `tool_calls` array. This is the join key. It is present on every fragment. - **`id`** — present only on the first fragment for that index. - **`function.name`** — likewise typically arrives with the first fragment. - **`function.arguments`** — a *fragment* of the JSON string, arriving across many chunks. So the accumulation logic is: for each fragment, look up (or create) a slot keyed by `index`; set `id` and `name` if the fragment carries them; append `function.arguments` to the slot's buffer if present. Nothing else is safe to assume — in particular, do not key on `id` (it is absent on most fragments) and do not rely on chunks for different indices arriving in contiguous runs, since with parallel calls enabled fragments for several indices can interleave. ## When the call is complete The last chunk for the choice carries `finish_reason`. `"tool_calls"` means the model has finished asking; `"stop"` means it produced prose instead; `"length"` means it hit the token ceiling — and that last one matters more here than for text, because a truncated arguments buffer is unparseable JSON, not a shortened sentence. Treat `"length"` during a tool call as a hard error: raise the ceiling or shrink the tool surface, do not try to repair the JSON. Only after the stream closes do you `json.loads` each accumulated buffer. Then reassemble the assistant message — with the full `tool_calls` array — and append it to your transcript exactly as you would in the non-streaming case, followed by one tool message per `tool_call_id`. The streaming path changes only how you obtain the call, not the protocol for answering it. ## What you can show the user The practical reason to stream tool calls at all is UI feedback. The function *name* arrives almost immediately, well before the arguments finish, so an agent UI can render "Searching the knowledge base…" within a few hundred milliseconds instead of after the whole call materializes. Streaming the partial arguments to the user is usually a mistake — half a JSON object is noise — but the name and the count of calls are genuinely useful progress signals. ## Interleaving with text A turn can begin with prose and then switch to tool calls, or come back after tool results and stream ordinary text. Your consumer therefore has to handle both `delta.content` and `delta.tool_calls` on every chunk, rather than branching once at the start of the stream. In an agent loop this repeats per iteration: stream, accumulate, dispatch, stream again — and the final iteration usually ends `"stop"` with content, which is what you render as the answer. ## Errors mid-stream A stream can fail after chunks have already been delivered — a dropped connection, or a server-side error emitted partway through. You are then holding a half-built arguments buffer that must be discarded, not executed: a partially-parsed call is exactly the case where a `delete` with a truncated filter does damage. Make the accumulator's output atomic — either every call parses and you dispatch the batch, or you discard the whole assistant turn and retry the request. Retrying is safe because you have not yet appended anything to the transcript. ## Practical checklist - Key accumulation on `index`, never on arrival order or `id`. - Concatenate arguments; never parse a fragment. - Treat `finish_reason: "length"` during a tool call as fatal. - Handle content and tool_calls deltas in the same loop. - Discard, don't salvage, a truncated call on stream failure. - Once assembled, rejoin the normal path: append the assistant message, then one tool message per id.