In vLLM, which flags turn on OpenAI-style tool calling for a chat model?

## Why this is not just a switch The OpenAI-shaped API pretends tool calling is a property of the protocol: you send `tools`, and you get back `tool_calls`. Underneath, it is entirely a property of the *model*. A tool-calling checkpoint has been fine-tuned to emit a call in one specific textual form — one family wraps a JSON object in special tool tokens, another emits a tagged block, another writes what looks like a Python function call. There is no shared wire format. A server that wants to return a structured `tool_calls` array therefore needs two things it cannot infer: permission to accept `tool_choice: "auto"`, and a parser that knows this family's syntax. That is exactly the pair of flags. `--enable-auto-tool-choice` opts the server into automatic tool selection; `--tool-call-parser <name>` selects the extractor. They go together — enabling one without the other is a configuration error, and clients sending `tools` with `tool_choice: "auto"` to a server without them get a rejection rather than a silent degradation. ## What the parser actually does On a non-streaming request the parser runs over the finished generation, finds the tool-call region, and rewrites the response so that `choices[0].message.content` holds any surrounding prose (often null) and `choices[0].message.tool_calls` holds one entry per call, each with an `id`, a `function.name`, and `function.arguments`. Note that `arguments` is a **JSON string**, not a nested object — that is the OpenAI shape, and clients must parse it themselves. `finish_reason` becomes `"tool_calls"`, which is how a client loop knows to execute the tool rather than show the text to a user. Streaming is a separate, harder code path in the same parser: it must decide, incrementally, that the tokens arriving are the beginning of a call, and emit deltas carrying `tool_calls[].index`, then the name, then fragments of the arguments string. Some parsers are more robust here than others, and streaming tool calls are a common place to find rough edges when a model family is newly supported. ## The chat template is half the feature A parser handles the model's *output*. The model's *input* — how the available tools are described to it — comes from the chat template, which receives the request's `tools` as a rendering argument and is responsible for laying them out in whatever form the checkpoint was tuned on. If a template ignores the `tools` argument, the tools never reach the model at all, and you will see the model answering in prose while your parser dutifully finds nothing to extract. When debugging a tool-calling setup, verify the rendered prompt first; the same `/tokenize` route that shows a chat prompt shows the tool block. ## The silent failure mode Mismatching the parser to the model is the classic mistake, and it does not raise. The server starts, requests succeed with a 200, the model happily emits its call in its own syntax, the parser looks for a syntax that never appears, and `tool_calls` stays empty while the raw call text sits in `content`. Downstream, an agent loop sees a normal assistant message and either loops or answers nonsense. Always test one tool-calling request end-to-end after any model change and assert on `finish_reason == "tool_calls"` rather than on the response merely arriving. The parser name must track the checkpoint, not the vendor. Two models from the same organization can use different formats across generations, and the set of parsers grows every release as families are added, so the correct value is a per-checkpoint fact you look up rather than remember. ## When parsing is the wrong tool If what you actually want is a guaranteed structure — a filled-in record rather than a genuine decision by the model to call something — do not rely on free-text parsing at all. Send `response_format` with a JSON schema so decoding is constrained to that grammar. Structure then comes from the decoder rather than from a post-hoc extractor, so it cannot be malformed, at the cost of the model no longer choosing whether to call. Use tool calling when the choice matters; use constrained decoding when the shape matters. ## Reasoning models Checkpoints that emit a separate thinking block complicate extraction, because the reasoning text may itself contain something that looks like a call. vLLM exposes `--reasoning-parser` to split that block into a `reasoning_content` field, keeping it out of the answer the tool-call parser inspects. If you serve a reasoning model with tools, configure both parsers, and expect the combination to need a real end-to-end test before you trust it.