A deployed assistant's HTTP endpoint answers with a JSON envelope rather than a bare string. What must garak's HTTP generator be told about that envelope, and what does a scan report look like when that setting is wrong?

### The contract the extraction setting satisfies Downstream of the generator, garak deals only in strings. Everything that makes an HTTP reply structured — status, conversation and message ids, role labels, tool calls, citations, safety metadata, token usage, streamed chunks — must be collapsed by the adapter into one message string per generation. That collapse is lossy, and it is yours to specify. In garak's REST generator config it is two keys: `response_json` says the body is JSON rather than raw text, and `response_json_field` says which part of it is the assistant's message — a top-level key name, or a JSONPath expression such as one selecting the last element of an `output` array and its `content`. If the endpoint streams, the generator must also accumulate chunks into a whole message before returning. ### What it costs to get wrong Nothing, at run time — and that is the problem. A wrong `response_json_field` costs you the entire sweep: the full request count, the full completion bill, and the full wall-clock, all spent producing a corpus that cannot be triaged into correctness afterwards. On a 15,000-request run against a metered endpoint that is hours and a real line item, discovered only when someone finally reads a stored reply. There is no partial recovery: detectors cannot be retuned to fix text the model never produced. The run is redone. ### How the number misleads, failure mode by failure mode - **Whole envelope stored.** Substring and pattern detectors now match against JSON key names and metadata as well as content. You get false hits where a key or a role label coincides with a matched term, and false misses where the real text is present but re-encoded or escaped. - **Empty or null field.** Every attempt is an empty string, every detector abstains, and the report is beautifully clean. A sweep of a chatty production assistant that returns near-zero hits on *every* probe family is far more likely to be a broken selector than a uniquely robust deployment — uniform cleanliness across unrelated probes is the tell. - **Wrong turn selected.** If the path lands on the echoed user turn rather than the assistant turn, garak is scoring your own probe text. Detectors that look for markers of the attack itself then fire on nearly every attempt, and the deployment looks catastrophically broken. - **First streamed chunk only.** Replies are truncated at a fixed prefix. Long-form violations that appear after a polite preamble are never seen, so the run under-reports exactly the failures that matter most. - **Non-2xx bodies stored as replies.** Throttle and gateway-error payloads enter the corpus as though the model had said them. They contain no violation, so they read as passes and dilute the rate. Notice the direction: three of the five bias the number *downward*, toward "safe". A generator misconfiguration is a systematically optimistic error, which is why it survives review — nobody interrogates a good result. ### Why this is a generator problem, not a detector problem A detector's false-positive rate is a property of the detector, argued on its own terms and triaged case by case. This failure is upstream of all of that: the detector is behaving correctly on a string that never came from the model. No threshold, no detector swap and no post-hoc triage repairs it, because the evidence itself is wrong. ### What you check, in minutes Run one benign prompt through the configured generator, then open the run's JSONL log and read the stored output verbatim. It must be byte-identical to what a human user sees in the product. Repeat with a prompt whose answer you can predict, so you can tell a real reply from a plausible-looking wrapper. Repeat once more with a prompt that provokes a long answer, to prove the stream is accumulated rather than truncated. Finally, force one throttle or error and confirm you know where that body ends up. Only then is a multi-hour sweep worth launching.