skip to content

AI Products

Cloudflare runs model inference on its GPUs, stores vectors, proxies calls to other AI providers and hosts stateful agents. Interviewers ask how the pieces form a RAG or agent stack.

on this pageshow

explore

questions

23

In Cloudflare's Agents SDK, how does routeAgentRequest() reach an Agent instance, and why is the instance name the agent's identity?

level: juniorimportance: must knowfreq 30%

answer

  1. an agent is a Durable Object
  2. fixed URL path scheme
  3. kebab-case class, then instance
  4. same name, same SQLite database

basics

~20 s

routeAgentRequest() parses /agents/{class-in-kebab-case}/{instance-name} and forwards the request to the Durable Object for that name. The same class and name always reach the same agent and its private SQLite state, so the name is its identity.

solid answer

~40 s

An `Agent` from the `agents` package is a Durable Object, so it needs an exported class, a `durable_objects` binding and SQLite storage (the `exports` map with `storage: "sqlite"`, or the legacy `new_sqlite_classes` migration, never both). In `fetch`, `routeAgentRequest(request, env)` reads `/agents/support-agent/customer-42`, finds the binding for `SupportAgent`, gets or creates the instance named `customer-42` and hands it the request: WebSocket upgrades go to `onConnect()`, HTTP to `onRequest()`. It returns `undefined` when nothing matches, so you fall through to your own 404. Because the same class plus name always reaches the same instance with its own database, the name decides what the agent remembers: per customer, per room or one singleton. The name is an address, not proof of who the caller is.

code

typescript · 21 lines
typescript
import { Agent, routeAgentRequest } from "agents";

type State = { openTicket: string | null };

export class SupportAgent extends Agent<Env, State> {
  initialState: State = { openTicket: null };

  async onRequest(request: Request): Promise<Response> {
    return Response.json({ customer: this.name, state: this.state });
  }
}

export default {
  async fetch(request: Request, env: Env) {
    // /agents/support-agent/customer-42 -> SupportAgent instance "customer-42"
    return (
      (await routeAgentRequest(request, env)) ??
      new Response("Not found", { status: 404 })
    );
  },
} satisfies ExportedHandler<Env>;

go deeper

for a junior

Recall the URL scheme /agents/{kebab-class}/{name}, that an agent is a Durable Object, and the three config pieces: exported class, binding and SQLite storage.

for a middle

Explain that the name is the address of one Durable Object with private SQLite, and how naming per customer, per room or as a singleton decides what the agent remembers.

for a senior

Show that routing does not authenticate, and that you check the caller in onBeforeConnect or derive the instance name from the session before forwarding.

for a principal

Treat instance naming as data modelling: the name fixes the unit of isolation, data location and contention, and it is hard to change once state lives under it.

## What an agent is on Cloudflare An **agent** built with Cloudflare's Agents SDK is a class you write that extends `Agent` from the `agents` npm package. Under the hood that class is a **Durable Object**: the layering is `DurableObject` > `Server` (from the `partyserver` package) > `Agent`. That one fact explains most of the SDK. Every agent instance has a globally unique identity, runs single-threaded, and owns its own embedded SQLite database, which is where its state, its schedules and (for chat agents) its messages live. Because it is a Durable Object, you never construct an agent with `new`. The Workers runtime creates or wakes the instance when a request, an RPC call, a WebSocket message or an alarm arrives for it. ## The wiring: class, binding and storage Three things must line up before any request can reach an agent: 1. **The class is exported** from the Worker's entry module, for example `export class SupportAgent extends Agent<Env, State> {}`. 2. **A Durable Object binding** names it in the Wrangler file: `durable_objects.bindings` with `name` (the property on `env`) and `class_name` (which must match the exported class exactly). 3. **The class gets SQLite storage**. Current docs show two ways: the declarative `exports` map with `{ "type": "durable-object", "storage": "sqlite" }`, or the older `migrations` array with a `new_sqlite_classes` entry. They are mutually exclusive; Wrangler refuses a file that declares both. The SDK's configuration page also lists the `nodejs_compat` compatibility flag as required for agents. ## How `routeAgentRequest()` finds the instance In the Worker's `fetch` handler you call `routeAgentRequest(request, env)`. It reads the URL path, which follows a fixed scheme: ```plaintext /agents/{agent-class-in-kebab-case}/{instance-name} ``` | URL | Agent class | Instance | | --- | --- | --- | | `/agents/support-agent/customer-42` | `SupportAgent` | `customer-42` | | `/agents/support-agent/customer-7` | `SupportAgent` | `customer-7` | | `/agents/support-agent/default` | `SupportAgent` | `default` | The router converts class names to kebab-case and also accepts the original spelling. It finds the matching binding in `env`, gets or creates the Durable Object for that name, and forwards the request: a WebSocket upgrade goes to the agent's `onConnect()` and `onMessage()` handlers, a plain HTTP request to `onRequest()`. - When a route matches, it returns a `Response` (including the `101` for a WebSocket upgrade). - When nothing matches, it returns `undefined`, so your code falls through to its own routes or a 404. - A common bug is calling it without returning its result, or returning your 404 before calling it. From server code you can skip the URL entirely: `getAgentByName(env.SupportAgent, "customer-42")` returns a stub whose public methods you call over Durable Object RPC. ## Why the name is the identity The instance name is not a label on a shared object; it **is** the address. The same class and the same name always reach the same instance, wherever in the world the request starts, and that instance's SQLite database is private to it. So: - **Per-customer agents**: naming the instance after the customer (`customer-42`) gives every customer an isolated agent with its own history and schedules. - **Shared agents**: naming it after a team queue or a room makes everyone who connects share one instance and see the same synced state. - **A singleton**: one fixed name, such as `default`, which is also what `useAgent()` uses when you pass no `name`. Choosing the name is therefore choosing the unit of memory. A support agent keyed by customer remembers that customer across days; one keyed by a random session ID forgets them as soon as the tab closes. ## What the name does not do The name arrives in the URL, so it is an address, not a credential. Anyone who can reach the Worker and guess a name can try to connect to it. Production agents check who the caller is before routing, with the `onBeforeConnect` and `onBeforeRequest` hooks of `routeAgentRequest()` or with server-side routing that derives the name from the caller's session. That hardening is a topic of its own; the junior-level point is to know that routing alone does not authenticate. ## Common mistakes in a first agent - **Constructing the class yourself.** `new SupportAgent()` creates a plain object with no storage and no identity; only the binding (through `routeAgentRequest()`, `getAgentByName()` or a client connection) reaches the real instance. - **Forgetting SQLite storage.** The state, schedule and chat features all write to the agent's SQLite database, so a class declared without SQLite storage cannot use them. - **Swallowing the router's result.** `await routeAgentRequest(request, env);` without `return` drops the response, and the request falls through to your 404. - **Choosing a throwaway name.** If the client invents a new name on every page load, every visit lands on a fresh, empty agent, and the customer seems to be forgotten.

  • How does server-side code in the same Worker call a method on one customer's agent without building a URL?
    `getAgentByName(env.SupportAgent, "customer-42")` returns a Durable Object stub for that instance, and you call its public methods over RPC, for example `await stub.closeTicket(id)`. No `@callable()` decorator is needed for calls from the same Worker; that decorator exposes methods to WebSocket clients such as a browser.
  • What should you check first when every request to /agents/support-agent/customer-42 returns 404?
    That the class is exported from the entry module, that `class_name` in the binding matches it exactly, that the path uses the kebab-case class name, and that the `fetch` handler returns the result of `routeAgentRequest()` before its own 404. The SDK's error for a missing namespace lists the agents it can see.

  • Creating an agent with new SupportAgent() inside the fetch handler.
  • Expecting routeAgentRequest() to return a 404 Response when no agent matches.
  • Naming instances with a random ID per tab and expecting the agent to remember the customer.
  • Declaring both an exports entry and a new_sqlite_classes migration for one class.
  • Treating the instance name in the URL as proof of who the caller is.

Assumes Wrangler 4.149.0 as of 2026-10-10: the declarative exports map is the newer way to give an agent class SQLite storage; the legacy migrations array with new_sqlite_classes still works, but not in the same file.

open as a page

In Workers AI, how does a Worker call a model through the AI binding with env.AI.run(), and what comes back?

level: juniorimportance: must knowfreq 45%

basics

~20 s

A Worker declares an ai binding in its Wrangler file and calls env.AI.run(model, inputs) with an @cf/ catalogue id; the promise resolves to that task's output, such as { response } for text generation or { shape, data } for embeddings.

open as a page

In Cloudflare AI Gateway, how is response caching turned on, and what do cf-aig-cache-ttl, cf-aig-skip-cache and cf-aig-cache-key each change?

level: middleimportance: must knowfreq 32%

basics

~20 s

Caching is off until a gateway gets a default TTL or a request opts in. cf-aig-skip-cache bypasses the cache, cf-aig-cache-ttl sets how long a cached response lives, and cf-aig-cache-key replaces the exact-match key and opts that request in.

open as a page

In Cloudflare Vectorize, why are an index's dimensions and distance metric fixed at creation, and how do you choose them?

level: middleimportance: must knowfreq 32%

basics

~20 s

A Vectorize index is built for one vector length and one scoring function, so wrangler vectorize create fixes both. Copy the dimensions from the embedding model (768 for bge-base), use its intended metric, and create a new index when the model changes.

open as a page

In Cloudflare AI Gateway, how does an application route its existing model-provider calls through a gateway, and what does the gateway add?

level: juniorimportance: should knowfreq 30%

basics

~10 s

Point the provider SDK's base URL at gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}, or use the newer REST API on api.cloudflare.com. Each call is then logged and can be cached, rate limited and retried without rewriting the request body.

open as a page

With Cloudflare Vectorize, how does a Worker embed a search phrase with Workers AI and query an index for the closest pages?

level: juniorimportance: should knowfreq 30%

basics

~20 s

Bind the index under vectorize and Workers AI under ai, embed the phrase with env.AI.run() using the model that embedded the pages, then pass its data[0] vector to the index's query() with topK and returnMetadata.

open as a page

With Cloudflare's Agents SDK, how do you expose a support agent's tools to other agents as a remote MCP server now that McpAgent is deprecated?

level: middleimportance: should knowfreq 18%

basics

~10 s

Serve the tools from a Worker route with createMcpHandler(createServer) from agents/mcp/server, which builds a fresh MCP server per request; McpAgent is deprecated and kept only for migrating servers. Other agents connect with this.addMcpServer().

open as a page

In Cloudflare's Agents SDK, how does this.schedule() send a follow-up two days later, and what happens to it across hibernation and failures?

level: middleimportance: should knowfreq 25%

basics

~20 s

this.schedule(172800, "sendFollowUp", payload) stores a row in the agent's SQLite and arms its Durable Object alarm, so the agent wakes in two days even with nobody connected. Failed callbacks retry, then a one-time schedule is removed.

open as a page

In Cloudflare's Agents SDK, how do this.setState() and this.sql differ, and how does agent state reach a React dashboard using useAgent()?

level: middleimportance: should knowfreq 28%

basics

~20 s

setState() replaces one JSON state value, saves it to the agent's SQLite and broadcasts it to every connected client; this.sql runs your own queries on the same database and broadcasts nothing. useAgent() receives each broadcast as agent.state.

open as a page

With Cloudflare AI Gateway, how do an authenticated gateway and stored provider keys (BYOK) keep model-provider API keys out of application code?

level: middleimportance: should knowfreq 26%

basics

~20 s

An authenticated gateway accepts only requests carrying a Cloudflare token (cf-aig-authorization on gateway.ai.cloudflare.com), and BYOK stores the provider key in the gateway, so services hold a revocable Cloudflare token instead of the provider key itself.

open as a page

In Cloudflare Vectorize, how do insert() and upsert() differ for an existing vector ID, and when can a query see the write?

level: middleimportance: should knowfreq 26%

basics

~20 s

insert() keeps an existing vector and skips the new one; upsert() replaces it in full, metadata included. Both return a mutationId and apply asynchronously, so a write is queryable seconds to minutes later, which describe() shows through processedUpToMutation.

open as a page

In Workers AI, how do you embed support tickets with a @cf/baai/bge model, and what about its output must stay fixed?

level: middleimportance: should knowfreq 30%

basics

~20 s

Call env.AI.run("@cf/baai/bge-base-en-v1.5", { text: [...] }) with up to 100 texts; it returns { shape, data } with one 768-number vector per text. The model and its pooling must stay fixed, because vectors from another model or pooling cannot be compared.

open as a page

In Workers AI, how do you stream a text-generation model's draft reply to the browser, and what does stream: true change?

level: middleimportance: should knowfreq 35%

basics

~20 s

Set stream: true in the inputs of env.AI.run(); it then resolves to a ReadableStream of server-sent events, which the Worker returns as a Response with content-type text/event-stream so the browser shows the draft as it is generated.

open as a page

In a Cloudflare AIChatAgent support chat, how does a half-streamed reply survive the customer's tab reconnecting, and a deploy evicting the agent mid-stream?

level: seniorimportance: should knowfreq 20%

basics

~20 s

AIChatAgent buffers stream chunks in SQLite, so a reconnecting tab replays them and continues live. An eviction kills the model connection; on restart durable recovery rebuilds the partial reply and, by default, continues it with continueLastTurn().

open as a page

In Cloudflare's Agents SDK the instance name comes from the request URL; how do you stop one customer from connecting to another customer's support agent?

level: seniorimportance: should knowfreq 22%

basics

~10 s

Authenticate before routing with routeAgentRequest()'s onBeforeConnect and onBeforeRequest hooks and check the caller owns the named instance, or derive the name from the session and forward with getAgentByName(). Then guard state and callable methods.

open as a page

A support assistant behind Cloudflare AI Gateway sets cf-aig-cache-key to a hash of each customer's question, and customers now see other customers' order details; why, and how do you fix it?

level: seniorimportance: should knowfreq 22%

basics

~20 s

cf-aig-cache-key replaces the default key, which hashed the full body, customer data included. Keyed on the question alone, every customer asking it gets the first answer. Skip caching personal prompts; key shared ones on everything the answer depends on.

open as a page

A Cloudflare Vectorize docs search filters on product, yet pages from the original import never match although they carry product metadata; why, and how do you fix it?

level: seniorimportance: should knowfreq 24%

basics

~20 s

Vectorize filters only through metadata indexes, and a metadata index covers only vectors written after it was created. The product index was added after the import, so re-upsert the older vectors, and create metadata indexes before loading next time.

open as a page

On the Workers Free plan, a Workers AI ticket summariser fails with 429 errors every afternoon and recovers next day; what limit is it hitting, and how do you fix it?

level: seniorimportance: should knowfreq 28%

basics

~20 s

It is spending the daily free allocation of 10,000 neurons: Workers AI then returns 429 with internal code 3036 until the reset at 00:00 UTC. Move to Workers Paid, which bills usage beyond 10,000 at $0.011 per 1,000 neurons, or cut neurons per call.

open as a page

As the lead for services calling two model providers and Workers AI through Cloudflare AI Gateway, how would you set gateways, caching, failover and spend controls?

level: principalimportance: should knowfreq 16%

basics

~20 s

Give each application and environment its own gateway, cache only non-personal prompts, keep provider keys in BYOK, use retries for blips and Dynamic Routing fallbacks for provider outages, and cap cost with spend limits split by user metadata.

open as a page

As the lead of a support-ticket product, how do you decide whether summaries and draft replies run on Workers AI or on an external model provider behind AI Gateway?

level: principalimportance: should knowfreq 20%

basics

~20 s

Decide per workload: high-volume summaries and embeddings usually fit Workers AI's catalogue models, billed in neurons and called through a binding, while draft replies go wherever an evaluation on real tickets says quality justifies the cost, with every call routed through one AI Gateway.

open as a page

In Workers AI, how do you run a LoRA adapter trained on your support tone, and what does Cloudflare require of the adapter?

level: middleimportance: nice to knowfreq 12%

basics

~20 s

Upload the adapter with wrangler ai finetune create against a LoRA-capable base model, then call env.AI.run() on that base with lora set to the finetune's name or id. The adapter is two exactly named files, under 300 MB, rank up to 32.

open as a page

Cloudflare AI Gateway Guardrails are set to Block a hazard category in responses, yet a streamed chat reply with that content reached users; why, and what do you change?

level: seniorimportance: nice to knowfreq 14%

basics

~20 s

Guardrails do not support streaming. On the REST API a streamed response is evaluated and logged but still delivered in full, so Block is not enforced. Send stream: false wherever a response block must hold, and handle errors 2016 and 2017.

open as a page

For a Cloudflare Vectorize docs search split by product and language, when do you partition with a namespace and when with a metadata index?

level: seniorimportance: nice to knowfreq 18%

basics

~20 s

Use a namespace for one fixed partition every query names, such as product: it needs no metadata index and is applied first. Use metadata indexes, at most 10, for fields filtered in combination, by several values or by range, such as language.

open as a page