In LLM tool calling, how are parallel tool results returned and paired to their calls?

## Why a turn can hold several calls A model that has decided it needs stock levels for twelve stores has no reason to ask for them one at a time. When requests are independent — none of them needs another's answer to be formed — the model can emit several tool-call blocks in a single assistant turn. This collapses twelve round trips into one, which is usually the single largest latency win available in an agent loop, because each avoided round trip removes a full model inference from the critical path. Whether a model does this depends on the model, the tools, and sometimes a provider switch that disables the behaviour. It is a capability, not a guarantee: correct code handles one call and twelve identically. ## The return contract Three rules govern the way back, and all three are frequently broken: 1. **All calls must be answered.** You cannot answer eight of twelve and continue. The model's turn is a set of outstanding requests, and the provider generally rejects a continuation that leaves any of them dangling. 2. **The answers go in one turn.** You do not make a model call per result. One following turn carries all twelve result blocks; then the model runs once, seeing every answer at the same time. 3. **Pairing is by id.** Each result block repeats the id of the call it answers. Position is not the join key. Completion order is not the join key. ## The id-pairing bug The failure worth remembering is a runtime that fires twelve `check_stock` calls concurrently, appends each result to an array as it returns, and sends that array back. The results are now in completion order — fastest store first — while the calls are in the order the model emitted them. If the ids are also written from the loop index, or omitted where the provider tolerates it, the Leeds stock level is presented as Brighton's. What makes this vicious is that nothing errors. The shapes are valid, the token counts are normal, and the model produces a fluent, confident, wrong answer. It surfaces as "the assistant occasionally reports the wrong store", which reads like a model quality problem and is actually a plumbing bug. The fix is mechanical: carry the id from the call into the result on the same object, never through a parallel array, and assert before sending that the set of result ids equals the set of call ids. ## What you may not assume about ordering - **Not execution order.** Parallel calls carry no sequencing information. If the model emitted a write and a read of the same record in one turn, there is no defined order between them, and there is no way to express one. - **Not dependency safety.** The model asked for these together because it judged them independent. If they are not — because two of them mutate shared state — that is a tool-design and concurrency problem on your side, not something the protocol resolves. - **Not result ordering in context.** The model reads all the results together; do not encode meaning in their sequence. ## Concurrency is your choice Nothing obliges you to run parallel calls in parallel. You may execute them sequentially and still return them in one turn; the protocol only describes the message shape. Running them concurrently is where the wall-clock win comes from, so most runtimes do — with a bounded worker pool, per-tool timeouts, and a rule that one failing call does not abort the batch. A call that fails still needs a result block; a missing one breaks rule 1. ## When parallelism is the wrong shape Parallel calls are for fan-out over independent items. They are not a substitute for a plan. If the work is genuinely sequential — look up an order, then refund it, then email the customer — the model should be emitting one call per turn so it can react to each result. And if the fan-out is very wide (hundreds of items), a conversational turn carrying hundreds of results is often the wrong instrument entirely; that is where invoking tools from code inside a sandbox starts to win, because the loop never enters the context window. ## What good implementations do Keep call and result together as one object through the whole pipeline. Validate the id sets match before sending. Log the pairing, not just the payloads, so a mismatch is visible in traces. And test with deliberately staggered latencies so completion order differs from call order in your test suite — the bug does not reproduce when every stub returns instantly.