In Workers AI, how does a Worker call a model through the AI binding with env.AI.run(), and what comes back?
answer
- a binding, not a token
- catalogue id as the first argument
- inputs shaped by the model's task
- response field for text generation
basics
~20 sA Worker declares an ai binding in its Wrangler file and calls env.AI.run(model, inputs) with an @cf/ catalogue id; the promise resolves to that task's output, such as { response } for text generation or { shape, data } for embeddings.
solid answer
~40 sThe Wrangler file declares `"ai": { "binding": "AI" }`, which puts a Workers AI client on `env.AI`; the binding carries the account's identity, so the Worker holds no API token. `env.AI.run(model, inputs, options?)` takes a catalogue id such as `@cf/meta/llama-3.3-70b-instruct-fp8-fast`, an inputs object shaped by the model's task (`messages` or `prompt` for text generation, `text` for embeddings) and an optional third object for how the call is made, such as `gateway`. Without `stream: true`, a text-generation call resolves to an object whose `response` field holds the text, with `usage` token counts beside it; an embeddings call resolves to `{ shape, data }`. Code outside Cloudflare reaches the same models through the REST endpoint `/accounts/{account_id}/ai/run/{model}` with an API token, and gets the output wrapped in `result`.
code
javascript · 13 linesexport default {
async fetch(request, env) {
const { ticket } = await request.json();
const result = await env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast", {
messages: [
{ role: "system", content: "Summarise this support ticket in three sentences." },
{ role: "user", content: ticket },
],
max_tokens: 200,
});
return Response.json({ summary: result.response, usage: result.usage });
},
};go deeper
Recall the binding entry, the env.AI.run(model, inputs) call with an @cf/ id, and that text generation's text sits in the response field.
Explain the three arguments: model id, task-shaped inputs and call options, and how the result's shape changes with stream, returnRawResponse or queueRequest.
Show you keep the REST token on servers, read usage to estimate neurons, and treat a misspelt or retired model id as an error to catch, not a fallback.
Frame the binding as the default path for Workers and the REST API as the escape hatch, and decide where model ids live so a catalogue change is one edit.
## What the AI binding is **Workers AI** runs models from Cloudflare's catalogue on Cloudflare's GPUs. A Worker reaches it through a **binding**: a Wrangler file entry that places a client object on the `env` parameter of the Worker's handlers. ```json { "name": "support-tickets", "main": "src/index.js", "compatibility_date": "2026-10-10", "ai": { "binding": "AI" } } ``` The binding name is yours to pick; `AI` is the convention, giving `env.AI`. Because the binding carries the account's identity, the Worker stores **no API token** for Workers AI, and there is nothing to leak from its code or its secrets. Two side effects are worth knowing: - `ai` is one of the Wrangler keys a named environment does not inherit, so a staging environment declares it again; - there is no local simulator for this binding, so calls made during local development still reach Cloudflare's service and count toward the account's limits. ## Calling `env.AI.run()` `env.AI.run(model, inputs, options)` returns a promise and takes up to three arguments: 1. **model**: a catalogue id string such as `@cf/meta/llama-3.3-70b-instruct-fp8-fast` or `@cf/baai/bge-base-en-v1.5`. The id is exact. A misspelt or retired id does not fall back to another model: the error table lists `5007` (No such model) and `3042` (Invalid model ID). 2. **inputs**: an object whose shape the model's **task** decides. Text generation takes `prompt` or a `messages` array, plus parameters such as `max_tokens`, `temperature` and `stream`. Text embeddings take `text`, a string or an array of strings. Speech recognition takes `audio`. 3. **options** (optional): how the call is made rather than what the model sees: `gateway` to send the call through an AI Gateway, `extraHeaders`, `returnRawResponse`, `queueRequest` for the asynchronous Batch API, and `rejectIfBusy` to fail fast when capacity is short. Each model's page, or `wrangler ai models schema <model>`, prints the exact input and output schema, which is the reference when a call returns a validation error. ## What comes back, by task | Call | Example model | Resolves to | |---|---|---| | Text generation | `@cf/meta/llama-3.3-70b-instruct-fp8-fast` | `{ response, usage }`, plus `tool_calls` when tools were offered | | Text generation with `stream: true` | the same | a `ReadableStream` of server-sent events | | Text embeddings | `@cf/baai/bge-base-en-v1.5` | `{ shape, data, pooling }` | | Any call with `returnRawResponse: true` | any | a `Response` object | | A batch with `queueRequest: true` | models with batch support | an acknowledgement carrying a `request_id` to poll | `usage` reports `prompt_tokens`, `completion_tokens` and `total_tokens`, which is the raw material for estimating the call's cost in neurons. ## Calling the same models from outside a Worker Code that does not run on Workers (a nightly job, a backend elsewhere) uses the **REST API**: - `POST https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/{model}` with `Authorization: Bearer <token>` and the same inputs as a JSON body; - the token is a Cloudflare API token; one made without the dashboard's Workers AI template needs both `Workers AI - Read` and `Workers AI - Edit`; - the model's output arrives inside the API envelope, `{ "result": { ... }, "success": true, "errors": [], "messages": [] }`, so the text is at `result.response`; - a Chat Completions-compatible surface also exists under `/ai/v1/` (for example `/ai/v1/chat/completions`), for clients built around that request shape. The token belongs on a server. A browser that needs a summary calls your Worker, and the Worker calls the binding. ## Mistakes interviewers listen for - Storing an API token as a Worker secret to call the binding: the binding needs none. - Treating a text-generation result as a string: the text is in `response`. - Putting call options inside the inputs: `gateway` and `rejectIfBusy` go in the third argument, while `stream` and `max_tokens` are model inputs. - Expecting the REST API to return the bare model output: it is wrapped in `result`. ## Handling failures in the ticket Worker The promise from `env.AI.run()` rejects when Workers AI cannot serve the call, so the summary endpoint wraps it in `try`/`catch` and maps the failure to a status the caller understands. The error table is the guide to which failures are worth a retry: | Internal code | HTTP | Meaning | Sensible reaction | |---|---|---|---| | `5007` | 400 | no such model or task | a bug: fix the model id | | `3036` | 429 | daily free neuron allocation used up | a budget event: alert, do not retry | | `3040` | 429 | capacity temporarily exceeded | retry with backoff | | `3007` | 408 | request timeout | retry once | Logging the internal code beside the HTTP status keeps the two 429s apart, which matters because one needs a retry and the other needs a plan or budget change.
- Why does a Worker not need an API token to call Workers AI, while a script elsewhere does?The `ai` binding is attached to the Worker by the platform and carries the account's identity, so `env.AI.run()` is authorised by the binding itself. A script outside Cloudflare has no binding, so it authenticates to the REST API with a Cloudflare API token that holds Workers AI permissions. Keeping that token server-side matters: in a browser it would let anyone spend the account's neurons.
- How do you find out exactly which inputs a Workers AI model accepts?Read the model's page in the catalogue or run `wrangler ai models schema <model>`, which prints its input and output JSON schema. `wrangler ai models list` with `--task` or `--search` narrows the catalogue first. The schema shows, for instance, whether a text model takes `prompt`, `messages` or both, and its `max_tokens` default.
- What does passing a gateway option to env.AI.run() change?A third argument such as `{ gateway: { id: "support" } }` sends the same Workers AI call through the named AI Gateway, which adds its logging, analytics and optional caching; `env.AI.aiGatewayLogId` then holds the log entry's id. The model, the inputs and the shape of the result do not change.
- The Worker must keep a Workers AI API token as a secret to call env.AI.run().
- A text-generation call to env.AI.run() resolves to a plain string.
- An unknown model id quietly falls back to a default catalogue model.
- The REST API returns the bare model output with no result wrapper.
- Options such as gateway belong inside the inputs object.