Before a garak engagement on a customer-facing assistant, you must choose where the generator points: the vendor model endpoint behind the app, a staging deployment of the app, or live production. How do you argue the choice, and what does each option's number actually mean?

### One config key, three different measurements The `uri` in garak's REST generator config is a scoping decision disguised as a URL. Each candidate answers a different question, with a different denominator, and a report that does not name which was scanned is not interpretable. | Target of `uri` | Question it answers | What it cannot say | |---|---|---| | Vendor model endpoint behind the app | How does the underlying model behave with no application scaffolding? | Nothing about shipped risk | | Staging deployment of the app | Does the product, as engineered, resist these probes? | Nothing about live gateway, quota or abuse controls | | Live production | Does the deployed system, with its real defences and operations, resist them? | Nothing cheaply, safely or repeatably | **Model baseline.** Removing the application removes its system prompt, retrieval corpus, tool surface and output filter from the measurement. That is precisely its value: it is an *attribution* instrument. A hit that reproduces at the bare model is the model's; a hit that appears only through the app implicates the app's own context, which usually means the fix is app-side. It is never quotable as product risk. **Staging.** This is where most of an engagement belongs, because it is repeatable and has no customer blast radius. Its entire validity is drift. A staging system prompt one revision behind production, a smaller or synthetic retrieval corpus, a guardrail disabled to make development bearable, a different model version pinned in config — each silently invalidates the number without changing anything you can see in the report. Diff those four explicitly against production and record the deltas alongside the result. **Production.** Uniquely includes the gateway, the real rate limit, abuse heuristics, alerting and whether anyone notices — none of which staging usually has. ### What production actually costs Thousands of attack prompts land in real conversation logs that humans and downstream analytics will read, and that may be retained under a policy nobody consulted. The sweep consumes production quota that customers are also using. The completions are billed to a live cost centre. On-call may be paged by your own traffic. And the account can be suspended partway through, which does not merely stop the run — it destroys its denominator. ### Where the number misleads The failure specific to production is the flattering truncation. When abuse controls start returning 403s or a WAF interposes a block page, those bodies are stored as replies unless you have handled them. They contain no policy violation, so detectors find nothing, and the failure rate *falls* over the second half of the run. The report reads as a system that got safer under sustained attack; what actually happened is that the tail of the sweep never reached the model. Distinguishing the two requires inspecting stored replies and HTTP statuses, not reading the rate. The staging failure is the opposite and quieter: a clean staging result quoted as the product's posture when the production prompt revision, filter configuration or retrieval content differs. And the model-baseline failure is the oldest one — a vendor-endpoint number presented as the shipped product's risk, which is a category error, not a rounding error. ### The defensible plan Run the bulk of the sweep against a staging build that mirrors production, with the mirroring verified rather than assumed. Then run a small, narrowly scoped production pass to confirm findings that matter, agreed in writing with the operators: a time window, a request ceiling, a kill switch, and a named contact who can abort. Authorisation attaches to the credential in `headers` and the `uri` it is used against — that pair is what defines the environment and tenant you touched, so scope the written permission to exactly that pair and keep the run log as evidence. ### What the report must state The endpoint class scanned; the app or prompt revision it was running; the credential's tenant; the probe families included and excluded; the total request count; and the wall-clock. Without those six, two scans of "the same assistant" are not comparable and neither can be re-run. If the run was cut short — by suspension, by a deploy, by quota — say so, and treat the suspension itself as a genuine finding about live abuse controls that staging could never have produced.