Tool & Function Calling

## What tool calling actually is Tool calling (historically "function calling") does not let the model execute anything. It is a structured-output protocol: you describe the functions your application can run, and instead of prose the model emits a machine-readable request naming one of them plus arguments. Every side effect — the HTTP call, the SQL query, the file write — happens in your process, under your auth and your validation. That split is the point of most interview questions on this topic. ## Step 1 — describing the tools On a Chat Completions request you pass `tools`, a list where each element is: `{"type": "function", "function": {"name": ..., "description": ..., "parameters": {JSON Schema}}}` `name` is the identifier you will dispatch on. `description` is prompt text the model reads — it is the single strongest signal for *when* to pick this tool, so write it for a reader, not as an afterthought. `parameters` is a JSON Schema object (`type: "object"` with `properties` and `required`); per-property `description` strings matter for the same reason. A function that takes no arguments still needs an object schema with empty `properties`. An optional `strict` flag on the function definition asks the API to constrain generated arguments to the schema rather than merely suggesting it. The whole `tools` block is serialized into the prompt, so it costs input tokens on every request that carries it. ## Step 2 — reading the response When the model wants a tool, the choice comes back with `finish_reason: "tool_calls"` and `message.tool_calls` populated; `message.content` is typically null. Each element looks like `{"id": "call_abc", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\":\"Oslo\"}"}}`. Two details trip people up. First, `arguments` is a **string** containing JSON, not a parsed object — `json.loads` it. Second, that string is model-generated: it can be malformed, can carry values outside your enum, and can name entities that do not exist. Parse defensively and validate against your own schema or model class before touching a database. Dispatch by looking `function.name` up in an explicit map of allowed handlers. Never resolve the name dynamically against your module namespace. ## Step 3 — feeding results back The follow-up request is the same conversation plus two kinds of new messages: 1. The assistant message you just received, appended **verbatim** with its `tool_calls` array intact. If you drop it, the tool messages have nothing to attach to. 2. One message per call: `{"role": "tool", "tool_call_id": call.id, "content": "..."}`. `content` must be a string, so serialize structured results with `json.dumps`. There is no `name` requirement — the `tool_call_id` is what binds the result to the request. Send that with the same `tools` list (the model may want another round), and the next response is usually ordinary prose that incorporates the tool output. ## What the API enforces Every `tool_call_id` in an assistant message must be answered by a tool message before the next completion, and the tool messages must follow that assistant message in the array. Omit one and the request fails with a 400 rather than degrading gracefully. Likewise a tool message with no preceding assistant `tool_calls` is rejected. This makes conversation history a small state machine you have to keep consistent — a common source of bugs once you start trimming old turns to control context growth, because trimming an assistant tool_calls message without its tool replies (or vice versa) corrupts the transcript. ## Errors and results are both just content If your handler throws, the idiomatic recovery is to put a short error description into the tool message content — `{"error": "city not found"}` — rather than raising out of the loop. The model can then apologise, ask the user, or retry with different arguments. Return the raw exception text only if it carries no secrets. ## Common mistakes - Treating `arguments` as a dict and getting an attribute error at runtime. - Forgetting to append the assistant message, so the tool replies are orphaned. - Passing a dict as tool `content` instead of a JSON string. - Returning the tool result straight to the user instead of letting the model phrase it — you lose the synthesis step the loop exists for. - Assuming the model called the tool because it was there: with the default `tool_choice: "auto"` it may just answer in prose, so always branch on whether `tool_calls` is present.