Prompt Converters

### The object and where it sits in the send path A PyRIT prompt converter is a subclass of `PromptConverter` with one required method, `convert_async(prompt, input_type)`, which returns a `ConverterResult` carrying `output_text` and an `output_type`. That is the whole contract: a value goes in, a rewritten value comes out. You attach converters to the thing that drives the run, never to the target — older PyRIT takes `prompt_converters=[...]` on the orchestrator, newer versions take a list of `PromptConverterConfiguration` on the attack's converter config. PyRIT's `PromptNormalizer` applies them on the outbound path immediately before the target's `send_prompt_async`, so the endpoint is handed a finished string and has no idea a chain existed. ### Two values are recorded, and that distinction is the whole leaf Every turn lands in PyRIT's memory as a `PromptRequestPiece` that stores `original_value` and `converted_value` as separate columns, alongside `original_value_data_type`, `converted_value_data_type` and `converter_identifiers`. The target received `converted_value` and nothing else. Any sentence you write afterwards that says "the prompt" is ambiguous until you say which of the two you mean, and that ambiguity is where most bad converter reports begin. ### Why a converter on its own is not an attack The objective — the thing the run declares success against — is configured on the attack and the scorer, not on the converter. A converter *carries* a request; it does not *make* one. Three consequences follow directly: - **Benign seeds stay benign.** If the underlying request asks for nothing a model should refuse, no chain, however exotic, turns the run into a finding. A hit there is by construction an artefact of scoring. - **The unit under test moves.** If the plain form is refused and the converted form is not, you have measured the *input path* — tokenisation, any input classifier in front of the model, the model's ability to read that form — not the model's behaviour on the request. Those are different claims with different owners, and conflating them makes a report wrong even when its counts are right. - **The technique is cargo here.** Whatever encoding or obfuscation a converter applies, its efficacy as an evasion belongs to the guardrail-evasion topic. What this leaf owns is that PyRIT composes such a transform in series, bills for it, and stores both sides of it. ### What it costs Local converters are microseconds of string work and cost nothing. A converter configured with a `converter_target` — anything that asks a model to paraphrase, translate, restyle or expand the prompt — is one full inference call per prompt per link, spent before the system under test is touched at all. Over a 200-seed corpus that is 200 extra calls for a single such link, several minutes of extra wall clock, and quota burned on an endpoint that is usually the same provider account as your scorer and your attacker model. The target call and a model-backed scorer's call are metered separately on top. So the honest unit of a converter run's cost is **calls per attempt**, never seeds. ### Where the number misleads Four readings go wrong, in ascending order of embarrassment. First, an input-handling result published as a model-alignment result: "we jailbroke the model" when what you showed is that a filter does not normalise a surface form. Second, the scorer is downstream of the transform — if the converted prompt nudges the model to answer in the same form, a substring or refusal-detecting scorer is reading text it was never calibrated on, and it fails in both directions: unreadable output flagged as compliance, genuine compliance in the converted form missed. Third, an aggressive rewrite can drift the request off the objective entirely, and the low success rate that follows is your chain's failure, not the target's robustness. Fourth, and purely mechanical: newer PyRIT accepts converters on the response path as well as the request path, and a chain configured into the response list changes nothing the target ever saw while still looking, in the config, like a tested chain. ### What I would check before believing anything Pull a dozen `PromptRequestPiece` rows out of memory and read `original_value` against `converted_value` side by side, confirming the converted string still asks for the objective at all. Check that `converter_identifiers` lists the chain you intended, in the order you intended — the cheapest available proof that the chain actually ran, on the request path. Keep one unconverted control run over the same seeds, scored the same way, so the converter's contribution is a measured difference rather than an assumption. And price a single prompt end to end before committing to the sweep.