Your data cannot leave your VPC — how do you run Cohere Command models?

## Why several channels exist at all Cohere's commercial centre of gravity is regulated enterprises — banks, insurers, health systems, governments — and those buyers frequently cannot send prompts containing customer data to a vendor's multi-tenant endpoint in another jurisdiction. Rather than lose that market, Cohere distributes the same Command weights through channels that put the inference inside a boundary the customer already trusts. This is a genuine differentiator versus vendors that only sell a hosted API, and it is why the question comes up in enterprise-facing interviews. ## The four channels **Cohere's own API.** `api.cohere.com`, bearer API key, newest models first, least operational work. The default unless something forbids it. **Hyperscaler managed catalogues.** Command models are offered through Amazon Bedrock, Amazon SageMaker, Azure AI Foundry and Oracle's generative AI service. Inference runs in the cloud provider's region you select, under your existing account, on your existing contract. For many enterprises this is the sweet spot: the data-residency and procurement boxes are ticked without anyone operating a GPU fleet. **Private / VPC deployment.** Cohere licenses container images that you run on your own GPU capacity — inside your VPC, or fully on-prem and air-gapped. Nothing leaves your network. You now own capacity planning, GPU procurement, autoscaling, upgrades and observability. Command A's design point — a flagship servable on as few as two high-end GPUs — is aimed squarely at making this affordable. **Open weights, for some models.** Cohere publishes weights for parts of the Command line for research and non-commercial use, which is useful for evaluation but is not a commercial deployment path on its own. ## What actually changes in your code This is the part interviewers want concretely. - **Authentication.** A Cohere API key works only against Cohere's endpoint. On Bedrock you use AWS credentials and signed requests, with IAM policies governing which models a principal may invoke. On Azure you use the endpoint's key or an Entra identity. On a private deployment you own the ingress and its auth entirely. Practically this means secret management, rotation and audit all move. - **Model identifiers.** Hosts namespace the models. Bedrock exposes ids shaped like `cohere.command-r-plus-v1:0`; Azure exposes deployment names you chose. Your config cannot carry one string across channels. - **The request surface.** Against Cohere's endpoint you call `/v2/chat`. Through a hyperscaler you go through that platform's inference API and its SDK, which has its own request/response envelope. Where the host exposes a Cohere-compatible endpoint, the SDK can be pointed at it — `cohere.ClientV2(base_url=..., api_key=...)` keeps your call sites identical — and that is worth checking for per channel, because it turns a rewrite into a config change. - **Feature and version currency.** New Command snapshots and new API capabilities ship on Cohere's platform first and reach marketplaces later, sometimes much later. Anything that depends on a brand-new field may simply not exist on the channel your compliance team chose. Verify feature availability per channel *before* you design around a feature. - **Billing and quotas.** Marketplace consumption is billed and rate-limited by the host, under its quota model, not Cohere's. Your cost dashboards, budget alerts and limit-increase requests all move to the host. ## Choosing, as an architect Start from the actual constraint, because teams routinely over-engineer this. "Data cannot leave our VPC" and "data must stay in the EU" are very different requirements: the second is usually satisfied by picking a region in a managed catalogue, at a fraction of the cost of running your own GPUs. Ask what the constraint really is, who owns it, and whether a contractual answer (a data-processing agreement, a zero-retention commitment) settles it. Private deployment is the answer when the requirement is genuinely network-level isolation or air-gap. Then price the total cost. Self-hosting converts a per-token variable cost into fixed GPU capacity plus an on-call rotation, an upgrade cadence, and a benchmarking practice. That trade wins at high sustained volume and loses badly at low or spiky volume, where idle GPUs burn money. ## Keeping options open Whatever you choose, put a thin internal interface in front of generation — a method taking your own request type, with per-channel adapters underneath handling auth, model-id mapping and response translation. Then a compliance decision, a regional expansion or a cost renegotiation changes an adapter and a config value, not every call site. Add a per-channel contract test that exercises the real endpoint, since the whole point is that the channels differ in ways unit tests with mocks will never reveal.