When is OpenRouter's unified API the wrong abstraction for a production app?

## What the abstraction actually buys A unified, OpenAI-shaped endpoint in front of many vendors gives you four concrete things: one credential instead of a dozen, one request shape so model choice becomes configuration, breadth for evaluation, and one integration to maintain instead of several. For early-stage products, for teams comparing models, and for applications that legitimately route different workloads to different models, that is a large win and it is why the pattern spread. ## The structural costs **Lowest common denominator.** A shared shape can only express what all vendors roughly share. Anything genuinely vendor-specific either gets a gateway-defined extension field or is simply unreachable. That is fine for chat, tools and streaming; it is limiting as soon as your value depends on a feature that lives outside the intersection. **Feature lag.** Vendors ship first into their own SDKs. The gateway then has to model the new capability, decide how it fits the unified shape, and expose it. That delay is unavoidable and it is worst exactly where you feel it most — at the frontier, where the new capability is why you wanted the model. **A hop and a dependency.** Every request now traverses an additional service. That is added latency, an added failure domain and an added attack surface, and your uptime is now a product of two availabilities rather than one. It also means your prompts — often the most sensitive payload your company sends anywhere — pass through a third party. **Flattened observability.** Normalisation collapses vendor-specific stop reasons and metadata into a common vocabulary. Good for portable code, worse for forensic debugging: the signal that would have explained a behaviour may not survive the mapping. **Commercial and compliance shape.** Enterprise buyers negotiate directly: data-processing terms, zero-retention commitments, regional data residency, capacity guarantees, security review, incident escalation. Those attach to a vendor relationship. An intermediary can offer its own terms, but it cannot hand you the vendor's, and for regulated workloads that is frequently the decisive argument. ## Signals that a workload has outgrown it - Your traffic is overwhelmingly one model, and has been for months. - You are writing workarounds for a capability the vendor supports natively. - Procurement, security or legal needs a contract with the model vendor specifically. - Latency budgets are tight enough that an extra network hop is material. - You need capacity commitments or throughput guarantees no reseller can give. - Debugging repeatedly stalls because the information you need was normalised away. ## Signals that it is still right - You are still choosing, and the cost of choosing wrongly is a rewrite. - Different workloads genuinely want different models — cheap and fast here, deep and expensive there. - You need optionality against a single vendor degrading, repricing or deprecating a model. - Your usage is small enough that per-vendor contracts are not worth the overhead. - You are shipping a product that lets *users* pick a model. ## The design that keeps both options open Do not let vendor choice be an architectural commitment. Put a narrow internal interface in front of generation — a function that takes messages, tools and a policy and returns a normalised result — and let both the gateway and a native SDK be implementations of it. That costs a little indirection and buys you the ability to move one workload at a time. It is also the only structure in which "we use the gateway" and "we call the vendor directly" can be true simultaneously, which is usually where mature systems land: the gateway for breadth, experimentation and long-tail models, a native integration for the one high-volume, deep-feature path that pays for its own code. ## Framing it in an interview The weak answer picks a side. The strong answer is a decision rule: use the unified API while breadth and optionality are worth more than depth, and migrate a specific workload — not the whole app — once its dependence on one vendor's capabilities, contracts or latency budget outweighs what a common shape can give it. Name the observable triggers, and name the interface that makes the migration a swap rather than a rewrite.