Agents & Model Clients

## What a model client is in AutoGen Agents in AgentChat do not talk to a provider directly. They hold a **model client** — an object with a uniform chat-completion interface — and swapping the client is how you swap models under an unchanged agent. Clients live in the extension package: `autogen_ext.models.openai` provides `OpenAIChatCompletionClient` and `AzureOpenAIChatCompletionClient`; `autogen_ext.models.anthropic` provides `AnthropicChatCompletionClient`. Because the client is a constructor argument, giving two agents different models is just passing two different client instances, and giving them the same model is passing the same instance — which is normal and desirable, since a client is a connection pool, not per-agent state. ## Why model_info exists AutoGen has to make decisions before it ever sends a request: can this model be given tool schemas at all? Can it accept image content? Can it be asked for JSON? What message conventions does its family use? For a recognized model name the client answers those from a built-in table. But the OpenAI client is deliberately usable against *any* OpenAI-compatible endpoint — a local inference server, a gateway, a vendor clone — through `base_url`. In that case the model name is meaningless to the table, and rather than guessing, the client requires you to declare capabilities via `model_info`. The fields: - `vision` — whether image content may be sent. - `function_calling` — whether tool schemas may be sent and tool calls expected. This is the one that matters most for agents: an `AssistantAgent` with tools is useless against a model where this is false. - `json_output` — whether a JSON-shaped response can be requested. - `family` — which model family this belongs to, used for family-specific handling; `"unknown"` is a legitimate value. - `structured_output` — present in recent versions, covering schema-constrained output. ## The dangerous failure mode A missing `model_info` gives you a loud error at construction time, which is the good case. A *wrong* `model_info` gives you a quiet, expensive one. Declare `function_calling: True` for a model that cannot do it and the agent will happily send tool schemas; the model responds with prose describing what it would call, no tool executes, and the agent returns a plausible-looking answer built on nothing. Declare `vision: True` and image content is sent to a text-only endpoint. Treat `model_info` as a contract you have actually verified against the endpoint, not a block copied from a tutorial. ## Azure wiring `AzureOpenAIChatCompletionClient` is a separate class because Azure's addressing is different: you name a **deployment** (`azure_deployment`) as well as the underlying `model`, and you must supply `api_version` and `azure_endpoint`. Authentication is either `api_key` or, preferably in production, `azure_ad_token_provider` for token-based auth so no static secret sits in configuration. Passing an OpenAI-style client with a rewritten base URL at Azure is a common mistake that produces confusing 404s on the deployment path. ## Lifecycle Clients are async and hold an underlying HTTP client, so close them: `await client.close()` when the application shuts down, or manage them with an async context manager. In a long-running service, build the client once at startup and share it across agents and sessions; building one per request leaks connections and adds handshake latency to every call. This is the inverse of the agent lifetime rule — agents are per-session state, clients are process-wide infrastructure. ## Streaming and other knobs The OpenAI client also takes ordinary request settings — `temperature`, `max_tokens`, `timeout`, `max_retries`, `parallel_tool_calls`. Note that token-level streaming for an agent is switched on at the *agent* (`model_client_stream=True` on `AssistantAgent`), which makes the agent emit streaming chunk events as the model produces them; the client must of course support streaming for that to work. Leaving it off means you still get the reply, just not incrementally. ## Interview framing The question behind the question is whether you understand that AutoGen cannot introspect an arbitrary endpoint. Capability declaration is a manual contract at the boundary, and the cost of getting it wrong lands as bad agent behaviour rather than a stack trace — which is exactly the class of bug that survives review and reaches production.