Which hidden tokens does an LLM chat request add beyond your message text?

## The model does not see a list of messages An API that takes an array of role-tagged messages is a convenience over something flatter. Under it, the conversation is serialized into one token sequence using a **chat template** — a fixed format the model was instruction-tuned on. The template inserts special tokens the tokenizer reserves for structure rather than content: a marker that a turn begins, an indication of the role, and a marker that the turn ends. Those special tokens exist precisely so the model can tell your instructions from the user's text and from its own prior replies, which is also why prompt-injection defence begins with never letting untrusted text forge them. ## The overhead is per message, and it compounds Roughly, the rendered request is: a prefix (system prompt, tool definitions, any provider preamble), then for each message a small constant of template tokens plus that message's content, and finally an opening marker inviting the assistant to speak. The constant is small — a handful of tokens — but it is charged once per message. One 2,000-character message pays it once. Two hundred ten-character messages pay it two hundred times, and the structure can then outweigh the content. This is the direct explanation for the most common support question in LLM apps: *why does the provider report more input tokens than I counted?* You counted content; you were billed content plus structure. ## Tool schemas are the biggest hidden item Tool and function definitions are not metadata. Their names, human-readable descriptions, parameter names, enums and nested JSON Schema are rendered into the prompt as text and re-sent on **every** request in the conversation, because the model is stateless across calls. A dozen well-documented tools can occupy thousands of input tokens before the user has typed a character, paid on every turn. Trimming verbose descriptions or deferring rarely used tools is one of the highest-leverage cost reductions available, and it is invisible to anyone who budgets by message length alone. ## Why your local count can disagree with the provider's Three causes, in order of frequency. You tokenized raw strings instead of the templated sequence, so you missed the per-message and prefix overhead. You used a tokenizer from a different model family, so the vocabulary differs. Or the provider adds something you did not author — a system preamble, a tool-use scaffold, an injected date. The fix is the same in all three cases: render through the correct template for the deployed model, tokenize with the matching tokenizer, and then reconcile against the usage numbers in the response. Treat the reported usage as ground truth and your estimator as an approximation whose error you monitor. ## Consequences for design Budgeting. If a trimming policy assumes the window holds the sum of the message lengths, it will overflow on long, chatty threads exactly when history is most valuable. Budget against the templated count. Turn granularity. Splitting an instruction into many small messages costs more than the same words in one, with no benefit to the model — the template overhead is real tokens. This argues for consolidating system-level instruction into one system message rather than sprinkling it across turns. History management. Because history is re-sent whole on each call, per-turn overhead accumulates quadratically over a long conversation: turn *n* pays for all *n* prior turns and their markers. Summarizing or dropping old turns is a token decision, not only a quality one. Structure integrity. Special tokens are reserved. Text that merely looks like a role marker is normally encoded as ordinary content by the tokenizer, which is the point — but any pipeline that concatenates untrusted text into the raw templated string, rather than passing it as message content, risks letting that text impersonate a role boundary. ## What to say in an interview State plainly that the request is a rendered sequence, not a list; that role and turn-delimiter tokens plus the system prompt and tool schemas are billed input; that overhead is per message so many short turns cost more; and that the way to count is to apply the template, then tokenize, then reconcile with the reported usage. That answer covers the mechanism, the cost consequence and the operational check.