How would you structure model selection across providers in a LangChain service?

## The mechanism first `init_chat_model` builds a chat model from a string rather than an import: `init_chat_model("openai:gpt-4o-mini")`, or `init_chat_model("gpt-4o-mini", model_provider="openai")`. The provider package must be installed, but nothing in your business logic mentions `ChatOpenAI`. That alone removes the most common coupling: provider class names scattered across dozens of call sites. For runtime selection, `configurable_fields` makes named fields settable per call through the `config` dict — so one instance can serve a request routed to a cheap model and the next routed to an expensive one, without rebuilding objects. `config_prefix` keeps those keys namespaced when several configurable models coexist in one chain. ## Tiers, not model ids The design that survives contact with a growing codebase is a small, named set of tiers owned centrally: - **fast/cheap** — classification, routing, extraction, guardrail checks. High volume, low stakes. - **strong** — synthesis, long-context reasoning, anything user-facing and quality-sensitive. - **fallback** — a different provider entirely, for outages and quota exhaustion. Features request a tier. The mapping from tier to concrete model lives in config, so re-pointing a tier is one change, reviewed by whoever owns cost and quality. Without this, model ids proliferate: a year in, nobody can answer "which models are we actually calling in production" and a deprecation notice becomes an archaeology project. ## Where the abstraction leaks The interview substance is knowing that a uniform interface is not uniform behaviour. Things that differ under the same `invoke()` call: - **Structured output.** Strict JSON-schema mode exists on some providers and not others; `"function_calling"` is broader but weaker. A schema that validates everywhere under one provider may be rejected or loosely honoured under another. - **Tool calling.** Parallel tool calls, argument fidelity and adherence to descriptions vary materially between models of the same nominal quality. - **Usage reporting.** Whether streamed calls report usage, and whether cached-input and reasoning tokens are broken out, is provider-specific — so your cost ledger can lose fidelity on a switch. - **Reasoning models.** They may spend large hidden output budgets and ignore some sampling parameters, changing both latency and price shape. - **Context window and truncation behaviour**, and what happens when you exceed it. - **Prompt sensitivity.** The same system prompt does not produce the same behaviour across model families; prompts are model-coupled artefacts even though the code is not. Because of these, treat a model change like a dependency upgrade with behavioural risk, not like a config typo fix. ## The gate: evaluation and rollout A credible answer includes how you would actually make a switch safely: 1. **Offline suite** — a fixed set of representative inputs with graded outputs per tier, run before any re-point, scoring correctness, schema-validity rate, tool-call accuracy and refusal rate. 2. **Shadow or canary** — route a small share of live traffic to the candidate and compare outcome metrics, not just latency and cost. Structured-output parse failure rate is an unusually sensitive early signal. 3. **Budgets** — per-tier cost and p95 latency targets, alerting on regression, so a cheaper model that triples retries is caught. 4. **Fast rollback** — because the mapping is config, rollback is a config change, which is the main operational payoff of the whole design. ## When to drop the abstraction Be willing to say where the framework stops earning its keep. If a feature depends on one provider's distinctive capability — a specific caching mechanism, a bespoke response format, a batch API with different economics — pinning that call to the provider SDK or provider-specific class is honest engineering. The failure mode to avoid is the middle ground: pretending to be portable while every prompt and schema is tuned to one vendor, so the switch that the architecture promises has never been tested and would not work. Equally, portability is a means, not a goal. If you have one provider, one model and no plan to change, `init_chat_model` still helps by keeping construction in one place, but building a routing layer for a hypothetical migration is speculative complexity. ## Interview framing Open with the mechanism in one sentence, spend most of the answer on the leaks and the evaluation gate, and close with a clear statement of when you would deliberately couple to a provider. That sequence shows you understand that model selection is a cost, quality and risk decision that happens to be expressed in configuration — which is exactly the judgment a principal is being asked for.