You lead a red team whose targets are mostly third-party hosted chat endpoints, with one or two open-weights models deployed in house. A senior engineer proposes buying GPU capacity and standing up a weight-handling process so the team can run gradient-guided prompt searches. How do you decide, and what recurring costs sit behind the hardware line item?

**Start from the portfolio, not from the technique.** The ceiling on this capability is set by how often the team is handed loadable weights, and nothing about the hardware raises that ceiling. Count the last year of engagements and ask how many delivered a checkpoint you could load and were licensed to run. That fraction is the maximum share of work the investment can ever touch. In the situation described — mostly third-party hosted endpoints, one or two in-house open-weights models — the fraction is small, and that alone argues for renting capacity per engagement rather than owning it. **Costs that are not the accelerator.** - *Weight custody.* A client's model weights are among the most sensitive artefacts a consultancy will ever hold: they embody the training investment and, with a fine-tune, sometimes the training data's fingerprints. Holding them means encrypted storage, controlled and logged access, a documented retention window, evidenced deletion, and an answer to "what happens if that machine is compromised". Some clients refuse on this basis alone, which makes custody a sales constraint as well as a line item. - *Legal review.* Open-weights licences carry use restrictions; client weights carry contract terms. Someone reviews both, per engagement, before a byte lands on disk. - *Non-amortising compute.* Every target is a fresh search, because the gradient belongs to one parameter tensor. Every checkpoint update invalidates the previous result. A client on a quarterly release cadence turns a one-off purchase into a standing re-run obligation. - *Rig-fidelity maintenance.* This is the hidden hour sink and the one most often absent from proposals. Keeping a local rig faithful to each serving stack — quantised build, chat template, system preamble, filters — is recurring engineer time per target, and it is the work that decides whether the run's output means anything. - *Utilisation and ageing.* Owned accelerators idle between engagements are pure loss, and they age against model sizes that keep growing; a card that comfortably held last year's targets shards this year's. **Where the number misleads.** Two figures typically decide this review and both can be constructed to flatter the rig. The first is *cost per confirmed finding computed over weight-holding engagements only*: it drops the idle months and the fixed cost carried while the rest of the portfolio ran query-only, so the rig looks efficient on a slice chosen after the fact. Compute it over the whole portfolio and the same rig usually looks marginal. The second is *depth asymmetry in the reporting*: two in-house models receive the deepest testing available while eighteen hosted targets receive query-driven testing, and unless every report says so plainly, a clean result on a hosted endpoint gets read as though it had survived a search that was never run against it. That reading is the real institutional risk of owning the capability, and it is a writing discipline, not a hardware problem. **Alternatives to price against.** | Option | What it removes | What it introduces | |---|---|---| | Rent capacity per engagement | Fixed cost, idle time, ageing hardware | Still full custody obligations; procurement latency per job | | Run the job inside the client's environment | Custody, storage and deletion entirely | Scheduling dependence, restricted debugging, slower iteration | | Skip the class; invest in query-driven capability | Custody and compute both | No white-box depth on the in-house models | **How to say yes well.** If the answer is yes, scope it: name the specific high-consequence in-house models it exists for, publish the fidelity standard the local rig must meet before any hours are spent, cap hours per engagement with an agreed stopping rule, require calibration before full allocation, and review utilisation across the *whole* portfolio after two quarters rather than across the engagements the rig happened to serve. And write the standing rule that every report states which targets received this depth of testing and which did not.