A phone-banking voice gate certifies each accept against an attacker perturbing the audio - what does that cost per call?

## The claim has an operating cost, and it is per prediction An empirical robustness evaluation costs money once, in an offline run. A sampling-based certificate is different in kind: the guarantee is produced **at inference time, for every decision you want to be covered**. If the voice gate on a phone-banking line is to say "this accept is certified", the certification work happens inside that call. ### Bill one: forward passes A certified decision requires many passes over noised copies of the same input rather than a single pass. The sample count is not a tuning detail; it is the currency the certificate is bought with. A larger sample supports a larger radius and a stronger confidence level; a smaller one narrows the radius, weakens the confidence, or pushes the decision into abstention. So "we halved the samples to fit the latency budget" is not a performance change, it is a change to the strength of the claim, and it has to be re-stated in the claim. At IVR scale this is straightforward arithmetic that surprises people anyway: the multiplier applies to every certified call, in a caller-facing turn where seconds are visible. Two mitigations exist and both are honest — certify only a subset of decisions (say, only high-value account actions) and serve the rest uncertified, or accept a smaller radius. What is not honest is running the cheap path and reporting the expensive path's number. ### Bill two: abstentions The procedure has three outcomes, not two: accept, reject, or abstain. Abstention happens when the vote margin is too thin to support a positive radius at the required confidence — the certificate cannot be issued, so no certified answer is returned. This is the mechanism that keeps the guarantee true; suppressing it by "just returning the leading class anyway" converts a certified system into an uncertified one while leaving the paperwork unchanged. On this system an abstention is not a blank. It routes the caller to a human agent, which means the abstention rate is a **staffing and handling-time forecast**. A 2% abstention rate and a 12% abstention rate are the same technical mechanism and completely different operations, and the number is not knowable from the certificate's radius alone — it has to be measured on representative traffic. ### Where the abstentions land The important operational point is that abstentions are not uniformly distributed. Thin vote margins occur on inputs the model finds ambiguous: poor line quality, background noise, short utterances, accents or voices under-represented in enrolment. So the population that gets handed to a human agent is skewed toward callers who are already the worst served. That is worth measuring per segment rather than reporting a single aggregate, for the same reason any aggregate hides where a cost landed. It is also the thing an operations owner will notice first, long before anybody reads the radius. ### The margins interact with the dial Radius, confidence, sample count and abstention rate are one system, not four independent settings. Want a larger certified radius at fixed sample count? Margins that used to certify no longer do, and the abstention rate rises. Want fewer abstentions at the same radius? Buy more samples, and pay latency. Want both cheaper? Lower the confidence level, and the claim itself weakens. There is no configuration that delivers a large radius, a strong confidence, few abstentions and one forward pass; recognising that trade as a single surface is most of the senior judgment here. ### What to check in a running deployment - Is the vote actually running in the serving path, or was it dropped for latency while the claim stayed in the document? - What is the sample count in production, and does it match the one the reported radius and confidence were computed with? - What is the measured abstention rate on live traffic, per caller segment, and who absorbs it? - Which decisions are certified and which are not, and is that split visible to whoever reads the robustness statement? - Is the radius operationally meaningful at all for this input type, or is it so small that no realistic manipulation lives inside it? That last question is the uncomfortable one. A certificate can be perfectly correct and still cover a region too small to matter for the threat you care about, in which case you are paying compute and abstentions for a sentence that is true and not useful.