What does a per-request compute ceiling on a transcription API buy against clients who maximise per-call work?

## What the ceiling actually does A cap on the work one call may consume — forward passes, decoding passes, compute units, wall-clock — is the correct first control, and it is worth being precise about what it achieves. Before the cap, the worst admissible request costs an unknown amount; the exposure is open-ended and nobody can state it. After the cap, the ratio between the worst admissible request and a median one is **finite, known, and a number somebody chose on purpose.** That change — from an accident to a decision — is most of the value, and it is why this control goes in first even though it does not close the issue. ## What it does not do It does not remove the asymmetry. An adversary who wanted maximum work per call now takes maximum-allowed work per call. If the cap sits at eight times the median, one attacker request is worth eight ordinary ones and still costs them one ordinary price. You have converted an unbounded multiple into a bounded one. That is a real improvement and it is not a fix, and a candidate who claims otherwise has misread the mechanism. It also does nothing about the two non-monetary costs — the expensive calls still occupy capacity, so other tenants still wait longer. ## Who actually pays for the ceiling This is the part that separates a thoughtful answer from a reflexive one. The cap does not discriminate between an adversary and a customer; it discriminates between **cheap inputs and expensive ones**. In a call-centre quality product the expensive inputs are noisy lines, poor handsets, distant microphones, heavy crosstalk, unusual accents. Those are not edge cases to be discarded — they are frequently the recordings the customer most needs transcribed, and often the ones tied to the outcomes the product is bought for. So a ceiling set close to the median quietly amputates the legitimate tail. Worse, it usually does so **invisibly**: the call returns a transcript, just a worse one, or a truncated one, and it appears as slowly worsening quality for a subset of customers rather than as an error anybody investigates. If you ship a ceiling, ship it with an explicit signal — a distinguishable error or a flagged result saying the input exceeded the compute limit — so the cost of the control is measurable instead of absorbed silently by the customers with the hardest audio. ## Choosing where it sits Set the ceiling from the distribution of **legitimate** work, not from the attack. Measure what real traffic consumes per call, pick a percentile you are willing to refuse or downgrade, and then look at *whose* calls fall above it before you ship. If the answer is "three enterprise accounts with recordings from a factory floor", the number is wrong and no amount of security reasoning makes it right. ## The control that changes the economics The ceiling bounds the exposure. What removes the asymmetry is fixing the mismatch that created it: the service bills in **audio minutes** and spends in **compute**. When the price tracks the resource, an input that costs eight times as much to answer is charged eight times as much, and the amplification stops being the operator's problem — it becomes a purchase. The residual risk is then capacity contention among tenants rather than a bill nobody expected, which is a smaller and much better-understood problem. That change is not free either: metered compute pricing is harder to explain, harder to forecast for the buyer, and may not fit the product's market. Which is exactly why this decision is a joint one rather than a security ticket. ## Why an admission check is not a substitute A tempting alternative is to predict the cost from the input and reject expensive submissions at the door. In a pipeline with data-dependent compute you generally cannot predict per-call cost reliably before running it — that is the whole reason the pipeline branches at runtime on a confidence signal instead of deciding up front. Cheap admission checks on obvious proxies such as duration are still worth having, but the control that actually binds is enforced during execution, because that is where the work happens. ## What an interviewer is listening for That you credit the ceiling with what it genuinely buys — a bounded, known, chosen ratio — and refuse it the credit it does not deserve. That you can say who absorbs the accuracy cost and insist the cost be made visible. That you set the threshold from legitimate traffic. And that you name the billing-unit mismatch as the thing that actually removes the asymmetry, while acknowledging that changing it is a product decision with its own price.