When does running Whisper's open weights yourself beat calling the hosted transcription API?

## The license question, settled first Whisper's model weights and reference code were released under the **MIT license**. There is no usage-count threshold, no acceptable-use rider gating commercial deployment, and no redistribution restriction of the kind that complicates some other open-weight families. That removes legal risk from the decision entirely: if you want to run it in your own datacentre, embed it in a desktop app, or ship it inside an appliance, the license permits it. So this is purely a cost, control and operations tradeoff. ## The model ladder you inherit Self-hosting means choosing a size, which the hosted endpoint decides for you. The family spans roughly `tiny` (~39M parameters), `base` (~74M), `small` (~244M), `medium` (~769M) and `large` (~1.55B, through the v3 checkpoint), plus English-only `.en` variants of the smaller sizes that outperform their multilingual counterparts on English audio. There is also `large-v3-turbo`, a pruned-decoder variant of about 809M parameters that runs far faster than full large with a modest accuracy cost — though it is a transcription-focused variant rather than a translation workhorse. That ladder is the real prize of self-hosting. A call-centre pipeline processing clean English telephony may be perfectly served by a small English-only model running many streams per GPU, at a fraction of the compute a large model needs. Conversely, multilingual archival audio may justify full large-v3. The hosted `whisper-1` endpoint gives you one operating point — OpenAI has described it as the large-v2 checkpoint — and no dial. ## The cost arithmetic Hosted transcription bills per minute of audio processed. Self-hosting bills per hour of GPU, whether or not audio is flowing. The crossover is a utilisation question: - Compute your monthly audio minutes honestly, including retries and any overlap you add when chunking. - Estimate throughput for your chosen model size on your chosen hardware — smaller models and optimised runtimes can transcribe many times faster than real time, so one GPU may cover a great deal of audio. - Compare the hosted bill against **fully loaded** self-hosted cost: instance hours, redundancy for availability, storage and queueing, plus the engineering time to build and keep the pipeline working. That last item is what most build-versus-buy analyses undercount. Self-hosting means you now own long-form chunking, batching, queue backpressure, model warm-up, GPU failure handling, and upgrades when a new checkpoint lands. It is a service, not a library call. The shape of the answer: spiky, low-volume, or unpredictable workloads favour hosted decisively, because you pay nothing while idle. Continuous high-volume workloads — an ingest pipeline running around the clock — are where fixed capacity starts to win, and where the win can be large. ## Where compliance overrides the arithmetic Sometimes cost is not the deciding factor. Recorded audio is often among the most sensitive data an organisation holds: patient consultations, legal proceedings, customer support calls carrying payment details, internal meetings. If your data-residency commitments, sector regulation, or customer contracts say the audio does not leave your boundary, self-hosting is not an optimisation — it is the only compliant option, and the GPU bill is simply the price of doing the work at all. Air-gapped and on-device deployments fall in the same category, and the smaller checkpoints exist precisely to make them feasible on modest hardware. ## Control you only get self-hosted Beyond model size, self-hosting exposes decoding behaviour the API keeps internal: the compression-ratio, log-probability and no-speech thresholds that govern hallucination filtering; whether previous-window text conditions the next window; temperature-fallback retry policy; and the long-form chunking algorithm itself, which removes the 25 MB upload constraint from your design entirely. If you have audio with characteristics the defaults handle poorly — heavy crosstalk, music beds, very long silences — that tuning surface can matter more than raw model size. You also gain the option to fine-tune on domain audio, and to pin a checkpoint forever so behaviour never shifts under you. Against that, you lose the hosted path's free upgrades and its operational maturity. ## A defensible decision process Start hosted. It is a few lines of code and it gives you real measurements: minutes per month, latency requirements, accuracy on your actual audio, and which failure modes bite. Then re-evaluate against three questions in order: *Are we allowed to send this audio out?* If no, self-host. *Is volume high, steady, and growing?* If no, stay hosted. *Do we need decoding control or a model size the endpoint does not offer?* If yes, self-host; if no, stay hosted and spend the engineering time elsewhere. A hybrid is often the right end state — self-hosted capacity sized for baseline load, with the hosted API as burst overflow and as a fallback when your cluster is degraded. That works precisely because the same model family sits behind both, so transcript quality does not visibly change when traffic shifts.