A support assistant behind Cloudflare AI Gateway sets cf-aig-cache-key to a hash of each customer's question, and customers now see other customers' order details; why, and how do you fix it?
answer
- what the custom key left out
- the body no longer separates callers
- one provider key for every user
- skip the cache for personal prompts
- key on everything the answer depends on
basics
~20 scf-aig-cache-key replaces the default key, which hashed the full body, customer data included. Keyed on the question alone, every customer asking it gets the first answer. Skip caching personal prompts; key shared ones on everything the answer depends on.
solid answer
~50 sThe default AI Gateway key hashes the provider, endpoint, model, the provider auth header and the **full body**, and the body carried each customer's order data in the system prompt, so different customers' requests did not collide. `cf-aig-cache-key` **replaces** that key: every customer who asks "where is my order?" now maps to one entry, and the first customer's personalised answer is replayed until the TTL ends (the gateway default, or five minutes when none is set). The provider auth header gives no separation either, because the backend uses one provider key for everyone, and `cf-aig-metadata` is not part of any key. Fix it by sending `cf-aig-skip-cache: true` on every call whose prompt holds customer data, and give a custom key only to generic prompts, built from everything the answer depends on: prompt version, model and normalised question. Then treat the replayed answers as a data exposure, using the logs to find who received them.
code
javascript · 26 linesasync function sha256Hex(text) {
const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(text));
return [...new Uint8Array(digest)].map((b) => b.toString(16).padStart(2, "0")).join("");
}
// Headers for one AI Gateway call. A prompt that carries customer data is never cached;
// a generic FAQ prompt shares one entry, keyed on everything its answer depends on.
async function aigCacheHeaders({ personal, promptVersion, model, question }) {
if (personal) return { "cf-aig-skip-cache": "true" };
const normalised = question.trim().toLowerCase();
const key = await sha256Hex(JSON.stringify([promptVersion, model, normalised]));
return { "cf-aig-cache-key": `faq:${key}`, "cf-aig-cache-ttl": "3600" };
}
export default {
async fetch(request) {
const url = new URL(request.url);
const headers = await aigCacheHeaders({
personal: url.searchParams.has("customer"),
promptVersion: "support-v7",
model: "openai/gpt-4.1-mini",
question: url.searchParams.get("q") ?? "",
});
return Response.json(headers);
},
};go deeper
Recall that a cached AI Gateway response is replayed to anyone whose request maps to the same key, so personal answers must not be cached.
Explain that cf-aig-cache-key replaces the default exact-match key and that metadata and a shared provider key do not separate users.
Diagnose the leak from the key design, stop it with skip-cache on personal prompts, bound its window by the TTL, and use tagged logs to find affected customers.
Set a team rule that every custom cache key is reviewed for what it covers and what the response depends on, with personal data uncached by default.
## What happened The assistant builds each request from a system prompt that contains the signed-in customer's recent orders, followed by the customer's question. To raise the cache hit rate, someone added `cf-aig-cache-key: <sha256 of the question>`. The first customer to ask "Where is my order?" got an answer naming their own parcel and address. AI Gateway stored it under that key, and every other customer asking the same question received that stored answer. ## Why the default key was safe and the custom key is not By default AI Gateway builds its **cache key** by hashing five parts with SHA-256: - the provider and the endpoint; - the model; - the **provider authentication header**; - the **full request body**, every message and parameter. Because each customer's order data sat in the body, two customers could only share an entry if their whole prompt, orders included, was identical. `cf-aig-cache-key` does not add to that key; it **replaces** it. Requests that send the same custom value share one cached response, whatever their bodies say. A hash of the question alone threw away the part of the request that made the answer personal. Two things that look like protection do not help: - **The provider auth header** is in the default key, but the backend calls the provider with one key (or a stored BYOK key) for every customer, so it is the same for all of them. - **`cf-aig-metadata`**, where the app tags each call with a customer ID, feeds logs, analytics and spend limits. It is not part of the default key and has no effect on a custom one. ## How long the leak lasts A custom-key request uses `cf-aig-cache-ttl` if sent, otherwise the gateway's default TTL, otherwise **five minutes**. Each entry keeps being served until that TTL ends. Changing the code so that requests stop computing the leaked keys stops new hits at once; the stored entries simply age out. ## The fix 1. **Stop caching personal prompts.** Any call whose body carries customer data sends `cf-aig-skip-cache: true` and no custom key, or goes through a gateway whose caching is off. An answer about one person's orders is not reusable. 2. **Cache only generic prompts, and key them completely.** For FAQ-style questions answered from public policy text, a custom key is fine if it includes everything the answer depends on: the system-prompt version, the model and the normalised question. Leaving the prompt version out would keep serving answers written under an old prompt after a release. 3. **Prefer the default key when in doubt.** Its exact match on the full body is conservative; a custom key should exist only where the team has written down why sharing is safe. 4. **Set an explicit TTL** matched to how long the generic answer stays right, rather than inheriting the gateway default or the five-minute fallback. 5. **Keep personal data out of shared prompts.** If a generic FAQ flow ever starts adding the customer's name or orders to its system prompt, it has become a personal flow and must leave the cached path. ## Handling the incident - Use the AI Gateway logs: an entry holds the request and the response, and the dashboard's log filters include cache status and metadata values, so the `cf-aig-metadata` customer tags on cached hits in the incident window show which customers received a replayed answer. - Treat it as a personal-data exposure under the organisation's process, not just a bug. - Add a review rule: a change that introduces or alters `cf-aig-cache-key` needs a reviewer to say what the key covers and what the response depends on. ## A related trap **DLP** scanning runs only on a cache miss, when the gateway forwards a request to the provider; a cache hit skips it. A response cached before a new DLP policy was added is served unscanned until its TTL expires, so tightening a policy after an exposure does not clean what is already cached; `cf-aig-skip-cache` forces fresh, scanned responses in the meantime. | Request carries | Safe cache choice | | --- | --- | | Customer data in the prompt | `cf-aig-skip-cache: true`, no custom key | | Generic question, public context | Custom key over prompt version, model and normalised question, explicit TTL | | Unsure | Default key (exact match on the full body) or no caching |
- Would keying the custom cache key on customer ID plus question make the personalised answers safe to cache?It stops cross-customer leaks, but the answer reports live order status, so a cached reply can be wrong within minutes, and repeat questions from one customer are rare enough that the hit rate is low. Order answers are better left uncached; per-customer keys suit stable personal content, with a short explicit TTL.
- After the fix ships, do the leaked entries keep reaching customers?Not through the fixed code: personal requests send `cf-aig-skip-cache: true` and no longer compute the old question-only keys, so nothing looks those entries up. The entries stay stored until their TTL ends, which is why any old client or service still sending the old key must be found and updated too.
It is like a pharmacy filing filled prescriptions by drug name instead of patient name: the next person asking for that drug is handed a bag with someone else's name and address inside.
- A custom cache key is added to the default key, so the body still separates customers.
- Tagging calls with a customer ID in cf-aig-metadata keeps their cached answers apart.
- The provider auth header in the key gives each end user a separate cache entry.
- Adding a DLP policy cleans responses that are already cached.
- Caching is safe whenever the visible question text is identical.