Your PyRIT run is too slow, so you raise its concurrency. What does that change about the run's wall-clock time and about its cost?

Two quantities are being confused whenever someone says raising concurrency will "help with cost". A run has a fixed amount of **work** — a set of billed calls determined entirely by configuration: objectives, `max_turns`, converter fan-out, scorers attached. It also has an **arrival rate** — how fast those calls are issued. Concurrency is a knob on the second only. Spend tracks the first. ### Why the bill does not move Billing is per call and per token, not per unit of elapsed time. Running the same 6,500 calls over one hour instead of six hours issues the same 6,500 calls. Nothing is skipped, and there is no volume behaviour that rewards density. The only cost concurrency can add is work that fails and is repeated, and past saturation that term stops being negligible: a turn that generated an adversarial prompt and got a target response and then failed at the scoring step has already burned two of its three legs and stored nothing you can report. ### Where the parallelism actually is, and is not PyRIT exposes the knob in more than one place — a concurrency bound on the executor that fans multiple objectives out at once, and a per-target throttle such as `max_requests_per_minute` on the target object itself, which paces that endpoint regardless of how many workers sit above it. What neither can do is parallelise inside a multi-turn objective. That loop is a feedback loop: the adversarial chat composes turn N+1 from the target's response to turn N and the scorer's verdict on it. Turn N+1 does not exist until turn N has completed all three of its legs. The parallel units are therefore *across* objectives and seed prompts, and across the converter variants of a single turn — never the turns themselves. A run with three objectives and a 50-turn budget cannot be made fast by raising a worker count; its floor is roughly 50 sequential turns of latency no matter what you set. ### Where it stops helping Model the three legs as three queues with three service rates. Pipeline throughput is the minimum of the three, and every worker added past that minimum converts into queue depth rather than completions: throughput flat, per-turn latency rising. The non-obvious part is that the saturating leg is frequently **not** the system under test. A generously provisioned production target paired with a modest quota on your own adversarial model or judge model means you are throttled by your own tooling, and the natural-looking fix — move the scorer to something cheaper or self-hosted — changes what the run measures, not merely what it costs. ### Where it makes the result wrong, not just slow Two failure modes matter more than the wasted wall clock. First, **partially-completed turns bill the legs that succeeded** and produce no scored row, so a saturated run's cost-per-finding quietly worsens while its output does not improve. Second, **heavy concurrency changes the thing you are measuring.** A target under abnormal load may shed requests, degrade to a smaller served model, trigger platform-side throttling, or route through different capacity than it does under production traffic. A run that hammered the endpoint has tested a condition you did not intend to test, and any conclusion drawn about the deployed system's behaviour is now confounded with the load you generated. On someone else's production system that is also a courtesy and often a contractual problem, not only a measurement one. ### What to check, and what to change instead Diagnose per leg, never in aggregate. Instrument in-flight counts and latency separately for the adversarial chat, the objective target and each scorer; raise concurrency one step and watch which of the three has latency climbing while the others' utilisation falls. That leg is the bottleneck. Total run duration cannot distinguish the three and will send you to tune the wrong one. Then size concurrency to the tightest of the three quotas rather than to the machine's core count, keep the parallel unit the objective rather than the turn, and if the wall clock is genuinely the binding constraint, **cut work rather than raise arrival rate**: fewer converter variants, one scorer instead of two, a shorter turn budget, fewer objectives in this pass. Those reduce cost and time together, which raising concurrency never does. The catch is that they also change what the run covers, so the report must say which knob was turned — a shorter turn budget changes the findings and a higher worker count does not, and a reader who cannot tell the two apart cannot interpret a clean result.