You lead a red-team group whose engagements almost always grant only a metered text API for the system under test, and rarely model weights. Which automated jailbreak search capabilities do you standardise on, and what do you give up by not building the white-box one?

### Frame it as fixed versus variable cost, priced against your actual access distribution Not against the literature. If nearly every engagement grants a metered text key and nothing else, then the capabilities worth *owning* are the ones almost every campaign reuses, and the rare capability is worth *renting*. **Cheap breadth — template and mutation pools.** Low fixed cost, low per-run cost, fully reproducible, and it is the only class that still runs when a client forbids sending their traffic to any third-party model at all. Its ceiling is that it only finds what someone seeded. Its underrated value is as a **regression layer**: the same maintained pool, run on every engagement, gives you a comparison across clients and across time that no adaptive search can, because an adaptive search never runs the same experiment twice. **Adaptive depth — the attacker-model loop.** Moderate fixed engineering cost, real per-run cost on three metered endpoints (attacker, target, judge). It finds things nobody seeded, because it reads the deployment's own refusals and rewrites against them. For a text-only group this is where differentiated capability lives. **White-box gradient search.** High fixed cost — GPUs, the weights you rarely receive, and a scarce skill — and it applies only when a client ships open weights or when you deliberately sell surrogate-and-transfer work with a transfer rate clients accept. ### The asset that outlives every algorithm Whatever the search, something has to decide what counted, deduplicate near-identical hits so ten variants of one weakness become one report item, and store transcripts so a finding can be re-run in six months when the client says it is fixed. That **judging, deduplication and logging harness** is the compounding asset, and it is portable across all three classes. A group that builds three searches and no shared harness re-litigates the definition of "a hit" on every engagement, and cannot compare any two of its own reports. ### Where the numbers mislead - **Cross-engagement comparisons die the moment the pool changes.** "Client B had four times the findings of client A" is meaningless if the seed pool grew between the two runs. Version the pool and quote the version, or the number is an artefact of your own tooling. - **Volume metrics flatter.** "We ran 12,000 probes" is a spend figure dressed as a coverage figure. The meaningful denominators are attack surface reached and confirmed deduplicated findings per unit spend. - **Absence of white-box results reads as absence of that weakness class.** This is the specific misreading a text-only methodology invites, and it is on you to prevent it, not on the reader to infer it. - **A rented capability's cost is not the invoice.** Gradient search needs someone who can tell a plateaued optimisation from a broken one, a skill that perishes if unused. If you keep it in-house and use it twice a year, you are paying to keep a skill warm that will be cold anyway. ### What to say to the client, and what to check Write the scope into the methodology in plain words: testing was conducted through the deployed interface only, so the results characterise **the deployment** — model plus system prompt plus filters — rather than the model at its strongest. That is a legitimate and often correct scope, because it is what an external attacker faces; it is simply not a silent omission. Then check, on a cadence: what fraction of the last twenty engagements granted weights, and is it moving? Is the harness genuinely shared, or has each search grown its own scorer? Are pool versions recorded on every report? And is there a named trigger for reversing the decision — a sustained share of clients shipping open weights, or repeated demand for results only a white-box method can produce — so the choice is revisited on evidence rather than on whoever argues loudest this quarter?