skip to content

In Workers AI, how does a Worker call a model through the AI binding with env.AI.run(), and what comes back?

level: juniorimportance: must knowfreq 45%

answer

  1. a binding, not a token
  2. catalogue id as the first argument
  3. inputs shaped by the model's task
  4. response field for text generation

basics

~20 s

A Worker declares an ai binding in its Wrangler file and calls env.AI.run(model, inputs) with an @cf/ catalogue id; the promise resolves to that task's output, such as { response } for text generation or { shape, data } for embeddings.

solid answer

~40 s

The Wrangler file declares `"ai": { "binding": "AI" }`, which puts a Workers AI client on `env.AI`; the binding carries the account's identity, so the Worker holds no API token. `env.AI.run(model, inputs, options?)` takes a catalogue id such as `@cf/meta/llama-3.3-70b-instruct-fp8-fast`, an inputs object shaped by the model's task (`messages` or `prompt` for text generation, `text` for embeddings) and an optional third object for how the call is made, such as `gateway`. Without `stream: true`, a text-generation call resolves to an object whose `response` field holds the text, with `usage` token counts beside it; an embeddings call resolves to `{ shape, data }`. Code outside Cloudflare reaches the same models through the REST endpoint `/accounts/{account_id}/ai/run/{model}` with an API token, and gets the output wrapped in `result`.

code

javascript · 13 lines
javascript
export default {
  async fetch(request, env) {
    const { ticket } = await request.json();
    const result = await env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast", {
      messages: [
        { role: "system", content: "Summarise this support ticket in three sentences." },
        { role: "user", content: ticket },
      ],
      max_tokens: 200,
    });
    return Response.json({ summary: result.response, usage: result.usage });
  },
};

go deeper

for a junior

Recall the binding entry, the env.AI.run(model, inputs) call with an @cf/ id, and that text generation's text sits in the response field.

for a middle

Explain the three arguments: model id, task-shaped inputs and call options, and how the result's shape changes with stream, returnRawResponse or queueRequest.

for a senior

Show you keep the REST token on servers, read usage to estimate neurons, and treat a misspelt or retired model id as an error to catch, not a fallback.

for a principal

Frame the binding as the default path for Workers and the REST API as the escape hatch, and decide where model ids live so a catalogue change is one edit.

## What the AI binding is **Workers AI** runs models from Cloudflare's catalogue on Cloudflare's GPUs. A Worker reaches it through a **binding**: a Wrangler file entry that places a client object on the `env` parameter of the Worker's handlers. ```json { "name": "support-tickets", "main": "src/index.js", "compatibility_date": "2026-10-10", "ai": { "binding": "AI" } } ``` The binding name is yours to pick; `AI` is the convention, giving `env.AI`. Because the binding carries the account's identity, the Worker stores **no API token** for Workers AI, and there is nothing to leak from its code or its secrets. Two side effects are worth knowing: - `ai` is one of the Wrangler keys a named environment does not inherit, so a staging environment declares it again; - there is no local simulator for this binding, so calls made during local development still reach Cloudflare's service and count toward the account's limits. ## Calling `env.AI.run()` `env.AI.run(model, inputs, options)` returns a promise and takes up to three arguments: 1. **model**: a catalogue id string such as `@cf/meta/llama-3.3-70b-instruct-fp8-fast` or `@cf/baai/bge-base-en-v1.5`. The id is exact. A misspelt or retired id does not fall back to another model: the error table lists `5007` (No such model) and `3042` (Invalid model ID). 2. **inputs**: an object whose shape the model's **task** decides. Text generation takes `prompt` or a `messages` array, plus parameters such as `max_tokens`, `temperature` and `stream`. Text embeddings take `text`, a string or an array of strings. Speech recognition takes `audio`. 3. **options** (optional): how the call is made rather than what the model sees: `gateway` to send the call through an AI Gateway, `extraHeaders`, `returnRawResponse`, `queueRequest` for the asynchronous Batch API, and `rejectIfBusy` to fail fast when capacity is short. Each model's page, or `wrangler ai models schema <model>`, prints the exact input and output schema, which is the reference when a call returns a validation error. ## What comes back, by task | Call | Example model | Resolves to | |---|---|---| | Text generation | `@cf/meta/llama-3.3-70b-instruct-fp8-fast` | `{ response, usage }`, plus `tool_calls` when tools were offered | | Text generation with `stream: true` | the same | a `ReadableStream` of server-sent events | | Text embeddings | `@cf/baai/bge-base-en-v1.5` | `{ shape, data, pooling }` | | Any call with `returnRawResponse: true` | any | a `Response` object | | A batch with `queueRequest: true` | models with batch support | an acknowledgement carrying a `request_id` to poll | `usage` reports `prompt_tokens`, `completion_tokens` and `total_tokens`, which is the raw material for estimating the call's cost in neurons. ## Calling the same models from outside a Worker Code that does not run on Workers (a nightly job, a backend elsewhere) uses the **REST API**: - `POST https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/{model}` with `Authorization: Bearer <token>` and the same inputs as a JSON body; - the token is a Cloudflare API token; one made without the dashboard's Workers AI template needs both `Workers AI - Read` and `Workers AI - Edit`; - the model's output arrives inside the API envelope, `{ "result": { ... }, "success": true, "errors": [], "messages": [] }`, so the text is at `result.response`; - a Chat Completions-compatible surface also exists under `/ai/v1/` (for example `/ai/v1/chat/completions`), for clients built around that request shape. The token belongs on a server. A browser that needs a summary calls your Worker, and the Worker calls the binding. ## Mistakes interviewers listen for - Storing an API token as a Worker secret to call the binding: the binding needs none. - Treating a text-generation result as a string: the text is in `response`. - Putting call options inside the inputs: `gateway` and `rejectIfBusy` go in the third argument, while `stream` and `max_tokens` are model inputs. - Expecting the REST API to return the bare model output: it is wrapped in `result`. ## Handling failures in the ticket Worker The promise from `env.AI.run()` rejects when Workers AI cannot serve the call, so the summary endpoint wraps it in `try`/`catch` and maps the failure to a status the caller understands. The error table is the guide to which failures are worth a retry: | Internal code | HTTP | Meaning | Sensible reaction | |---|---|---|---| | `5007` | 400 | no such model or task | a bug: fix the model id | | `3036` | 429 | daily free neuron allocation used up | a budget event: alert, do not retry | | `3040` | 429 | capacity temporarily exceeded | retry with backoff | | `3007` | 408 | request timeout | retry once | Logging the internal code beside the HTTP status keeps the two 429s apart, which matters because one needs a retry and the other needs a plan or budget change.

  • Why does a Worker not need an API token to call Workers AI, while a script elsewhere does?
    The `ai` binding is attached to the Worker by the platform and carries the account's identity, so `env.AI.run()` is authorised by the binding itself. A script outside Cloudflare has no binding, so it authenticates to the REST API with a Cloudflare API token that holds Workers AI permissions. Keeping that token server-side matters: in a browser it would let anyone spend the account's neurons.
  • How do you find out exactly which inputs a Workers AI model accepts?
    Read the model's page in the catalogue or run `wrangler ai models schema <model>`, which prints its input and output JSON schema. `wrangler ai models list` with `--task` or `--search` narrows the catalogue first. The schema shows, for instance, whether a text model takes `prompt`, `messages` or both, and its `max_tokens` default.
  • What does passing a gateway option to env.AI.run() change?
    A third argument such as `{ gateway: { id: "support" } }` sends the same Workers AI call through the named AI Gateway, which adds its logging, analytics and optional caching; `env.AI.aiGatewayLogId` then holds the log entry's id. The model, the inputs and the shape of the result do not change.

  • The Worker must keep a Workers AI API token as a secret to call env.AI.run().
  • A text-generation call to env.AI.run() resolves to a plain string.
  • An unknown model id quietly falls back to a default catalogue model.
  • The REST API returns the bare model output with no result wrapper.
  • Options such as gateway belong inside the inputs object.