promptfoo

A deterministic assertion compares the output against a rule you wrote, so it is free, instant, and gives the same verdict on every rerun. An llm-rubric assertion sends the output plus your written criterion to a grading model, costing one extra paid call per case and returning a verdict that can shift between runs.

The transform picks the assistant's text out of the HTTP response body and hands that string to the graders. Without one, promptfoo grades whatever the raw body serialises to: the whole envelope, metadata included. Assertions then match against wrapper fields, so the report looks plausible but describes the envelope, not the reply.

promptfoo crosses every prompt with every provider for every test case, so 3 x 4 x 25 is 300 calls per run, before any model-graded check adds its own. The matrix multiplies rather than adds, so each extra provider or prompt buys a whole column you pay for on every run.

Each promptfoo red-team plugin stands for one harm class. Adding one makes the generator write adversarial test cases aimed at eliciting that harm, and it supplies the grading criteria used to judge the replies. A plugin you leave out generates nothing, so that harm is never attempted and never shows up in the report.

It supports one narrow claim: the target refused those harms when asked plainly, in a single turn, in the generator's own phrasing. It says nothing about the same asks under an adversarial framing or across several turns, which is where real failures live. Report it as passing the plain baseline, never as safe.

promptfoo's red-team mode reads that description twice: the generator writes attack cases from it, and the graders judge replies against it. A one-line description yields generic attacks that miss your app's real surfaces, and gives the graders no stated rule to fail an answer against, so weak replies get marked pass.

It sets how many adversarial cases promptfoo generates for each enabled plugin, so total volume scales with plugins times that count. A small default gives each harm class only a handful of tries, so a failure the app commits rarely can easily never fire, and the run then reports a clean pass.

Three variables moved, so the drop has three possible causes and any of them could be masking a regression in another. Rerun the same saved adversarial suite from one baseline, adding one change at a time, with the grader and target settings fixed. Only then does a delta name a cause.

Generation is itself a model call, so a fresh run produces different test cases. Committing the generated suite makes it a fixed artifact: runs become comparable and diffable, and a failure is reproducible. Regenerating every run measures a new suite each time, so month-over-month numbers move for reasons nobody can attribute.

promptfoo caches responses keyed by the request it sent. If the test cases and the provider settings did not change, a rerun replays stored answers instead of calling the new model, so the numbers still describe the old deployment. Keep the cache while iterating on graders; clear or disable it whenever the system under test changed.

It measures wording, not behaviour. A model can open with a polite refusal and then comply anyway, so the check passes an unsafe answer. It can also refuse in different words, or safely answer a benign case, and the check fails a fine output. You end up scoring phrasing.

Either side can. If the tool holds it, every request carries the full prior turns and the app must be stateless. If the app holds it, the tool has to read a session identifier out of the first response and send it back on each later request. Pick one; mixing them grades unrelated single turns.

You can conclude that on this run, for those three harm classes, at that suite size, nothing failed. You cannot conclude the assistant is safe. Harm classes with no plugin enabled generated no cases, so the report is silent about them. Silence is a missing denominator, not a pass.

Strategies do not add cases, they re-express them. Each strategy takes the cases the plugins generated and emits a transformed variant of every one, so the suite scales with plugins times strategies rather than growing by a fixed amount. Every scheduled rerun re-pays that multiplied count in target calls, grader calls and wall-clock time.

By default promptfoo generates red-team cases through its hosted generation service, so the description of your application leaves your machine. Treat it as text you would hand a vendor: no secrets, no customer records, no internal hostnames. There is a documented switch that turns remote generation off and generates locally instead.

Treat a spotless report as a wiring hypothesis first. Read the graded output of a few cases as text, confirm it is the reply and not the envelope or an error body, and run two controls: a case that must produce a substantive answer and one you know the app refuses. If neither behaves, the target is mis-wired.

Keep the pairing you actually ship on every run and rotate the rest, so each provider and each prompt appears somewhere without crossing them all. Then state the loss out loud: a rotated matrix says a failure happened, not which provider-and-prompt pairing owns it. Re-cross the full matrix before a release.

No. promptfoo's graders judge each reply against the description you supplied. If it never says what the assistant must refuse, who may ask, or what records it reaches, an off-policy answer breaks no declared rule and is scored a pass. Rewrite the description with roles, data and refusals, then re-run.

The pass rate stops being a trend and becomes a coincidence: a drop could be the edited prompt, a change in the model behind the provider, or a different generated case set. Pin each prompt as a versioned artefact, record which version every run used, and move one axis at a time.

A second provider tells you whether a weakness lives in the model or in your prompt. A second prompt variant tells you whether your wording is what is holding. If you ship on exactly one provider, add the prompt variant; if you are still choosing a model, add the provider. Either one doubles the run.

Work from what the application can actually do: its tools, the data it can reach, who talks to it, and what it must never say. Enable the harm classes it could genuinely commit. Enabling everything costs generation and inference on irrelevant classes and buries real failures in noise nobody triages.

It means the refusal is keyed to the surface form of the request rather than to its intent: change the framing and the same harm goes through. Pool it into one overall pass rate and that signal disappears, because the plain copies dilute the wrapped failures. Report per plugin and per strategy instead.

Treating each case as an independent try, the chance of missing every time is 0.95 to the tenth, about 60 percent. So a clean run of ten mostly reflects the sample size. To be roughly 95 percent sure of seeing a one-in-twenty behaviour you need about sixty cases.

A net delta hides composition. Fixing nine cases while breaking seven also reads as two points better, and the seven new failures may be worse than the nine fixed ones. Join the runs case by case, split the moves into newly-failing and newly-passing, and group them by plugin category and severity.

Multi-turn and adaptive strategies build follow-ups from the target's own replies, so the prompts actually sent differ per model even from identical seeds. The grader is a model and may score a new refusal style differently. Caching, truncation and errored cases also differ. Rule them out by diffing sent transcripts and re-running the baseline twice.

Check whether the graded population changed: case count and case identity, then whether responses were served from cache, then whether the grader or its judge model changed. Only after those hold constant does a target change explain the drop. Each run should record suite identity, case count, target identity and cache mode alongside the number.

Stop grading what a rule can decide. Put deterministic checks on everything with a signature and let a case that already fails one skip the grader. Keep model grading for the semantic residue only, cache verdicts for unchanged outputs, and run the graded tier on a schedule rather than on every commit.

It measured how often a grading model, reading your rubric, declined to object. That is not the same as safe. Before quoting it, read a sample of passing transcripts, check what the rubric text literally asks, confirm errored grader calls are not counted as passes, and state the denominator: cases this suite generated.

The cases share one conversation, so each is graded on history other cases wrote. That produces both false hits, where an earlier case primed the refusal or the compliance, and false passes, and it makes the run unreproducible. Prevent it by scoping a fresh session per case; detect it by rerunning serially and diffing outcomes.