skip to content

In Cloudflare AI Gateway, how is response caching turned on, and what do cf-aig-cache-ttl, cf-aig-skip-cache and cf-aig-cache-key each change?

level: middleimportance: must knowfreq 32%

answer

  1. off until you turn it on
  2. exact match on the whole request
  3. SHA-256 over provider, model, auth, body
  4. 60 seconds to one month
  5. a custom key opts a request in

basics

~20 s

Caching is off until a gateway gets a default TTL or a request opts in. cf-aig-skip-cache bypasses the cache, cf-aig-cache-ttl sets how long a cached response lives, and cf-aig-cache-key replaces the exact-match key and opts that request in.

solid answer

~40 s

AI Gateway caching is disabled by default; you turn it on per gateway (**Settings** > **Cache Responses**, or `cache_ttl` in the API) or per request. The default cache key is a SHA-256 hash of the provider, endpoint, model, the provider authentication header and the **full request body**, so only byte-identical requests hit. `cf-aig-skip-cache: true` fetches from the provider and ignores any cached copy. `cf-aig-cache-ttl` sets the lifetime in seconds for a request that already uses caching, from 60 seconds to one month. `cf-aig-cache-key` replaces the default key and opts the request into caching even on a gateway with caching off; its TTL comes from the header, else the gateway default, else five minutes. `cf-aig-cache-status` on the response says `HIT` or `MISS`. Only text and image responses are cached.

go deeper

for a junior

Recall that caching starts off, that cf-aig-cache-status reports HIT or MISS, and what each of the three cache headers is for.

for a middle

Explain the exact-match key: SHA-256 over provider, endpoint, model, the provider auth header and the full body, and why a single changed parameter misses.

for a senior

Show you know where caching quietly fails or surprises: streaming responses, volatile concurrent misses, entries outliving a policy change, and the five-minute TTL fallback for a custom key.

for a principal

Treat each TTL and custom key as a freshness and data-sharing decision, and say which traffic a team may cache at all.

## What the gateway caches AI Gateway's cache stores a **whole provider response** at Cloudflare and replays it for a later request that maps to the same **cache key**. A hit never reaches the provider, so it costs no provider tokens and returns far faster than generation. This differs from a model provider's own prompt caching, which reuses the processed prefix of a prompt but still generates a fresh answer; that mechanism is covered with prompt caching, and the two can be used together. Caching applies only to **text and image responses** and only to identical requests. It suits traffic with a limited set of prompts: a support bot whose first turn is one of a few fixed options, a nightly job that asks the same classification question, a product description generated once per SKU. A free-form chat rarely repeats a byte-identical body and gains little. ## Turning it on - **Per gateway**: in the dashboard, **AI** > **AI Gateway** > the gateway > **Settings**, enable **Cache Responses** and pick the default TTL. Through the API the same setting is the gateway's `cache_ttl`. The auto-created `default` gateway starts with caching off (a TTL of 0). - **Per request**: send `cf-aig-cache-key`, which opts that one request in even when the gateway default is off. - To check the result, read the `cf-aig-cache-status` response header: `HIT` means the gateway answered from cache, `MISS` means the provider was called. ## How the default key is built By default AI Gateway concatenates these parts and hashes them with **SHA-256**: 1. The provider, such as `openai` or `anthropic`. 2. The endpoint, the provider API path. 3. The model, such as `gpt-4o`. 4. The provider authentication header, for example the `Authorization` bearer token. 5. The **full request body**: messages, tools, temperature and every other parameter. Any difference in the body, even one space in a system prompt or a changed `max_tokens`, makes a separate entry. Headers outside this list, such as `cf-aig-metadata`, do not take part. ## The three per-request headers | Header | Value | Effect | | --- | --- | --- | | `cf-aig-skip-cache` | `true` | Bypass the cache: fetch from the provider without using a cached copy. | | `cf-aig-cache-ttl` | seconds | Set the lifetime of this request's cached response, for a request that already uses caching. Minimum 60 seconds, maximum one month. | | `cf-aig-cache-key` | a string | Replace the default key and opt this request into caching. Requests sending the same value share one cached response. | When `cf-aig-cache-key` is present, the TTL is the `cf-aig-cache-ttl` value if sent, otherwise the gateway's default TTL, otherwise **five minutes**. The older names `cf-cache-ttl` and `cf-skip-cache` still work but were renamed to the `cf-aig-` forms, which new code should use. From a Worker, the AI binding's gateway options `skipCache`, `cacheTtl` and `cacheKey` mirror these headers. ## Limits and surprises - **Cacheable request size** is 25 MB per request; the TTL ceiling is one month. - **Streaming responses are not cached by default**, so a chat client that always sends `stream: true` sees no hits. - The cache is **volatile**: two identical requests sent at the same moment can both miss, because the first response may not be stored before the second arrives. Do not build correctness on a guaranteed hit. - A cached response is replayed as it was. DLP scanning runs only on a miss, when the gateway calls the provider, so a DLP policy changed after caching does not apply to entries still within their TTL; `cf-aig-skip-cache` forces fresh responses for new requests in the meantime. - A custom key is a promise that every request sharing it deserves the same answer. If the response depends on anything the key leaves out, one caller can receive another caller's response. ## Choosing a TTL A TTL is a freshness decision, not a performance knob. Ask how long the answer stays right: a policy summary that changes monthly can live for days, a price explanation for an hour, anything personal not at all. Keep in mind that changing a model or a system prompt changes the body, so the default key naturally stops matching old entries after a prompt release; a custom key does not, unless the prompt version is part of it.

  • Why does an AI Gateway with caching on still show MISS for a chat UI that sends the same first message every time?
    Most likely the body is not identical or the response streams. The default key hashes the full body, so a timestamp, a conversation ID or a different `max_tokens` makes each request unique, and streaming responses are not cached by default. Compare two logged requests byte by byte, and consider a `cf-aig-cache-key` for the fixed first turn.
  • How does an AI Gateway cache hit differ in cost from a provider prompt-cache hit?
    A gateway hit replays a stored response and never calls the provider, so there are no provider tokens to pay for that request. A provider prompt-cache hit still sends the request and generates new output; it only makes the cached prefix cheaper and faster to process.
  • What does cf-aig-cache-ttl do on a request to a gateway whose caching is off and that sends no cf-aig-cache-key?
    Per the caching documentation, `cf-aig-cache-ttl` sets the duration for a request that already uses caching; the header that opts a single request in is `cf-aig-cache-key`. Send both when one request should be cached for a chosen time on a gateway whose default is off.

  • AI Gateway caches every response by default once a gateway exists.
  • The default cache key covers only the last user message, so rephrased prompts still hit.
  • cf-aig-cache-ttl accepts any value, including a few seconds or a full year.
  • A cache hit in AI Gateway still bills the provider for output tokens.
  • Two identical requests sent at once are guaranteed to share one provider call.