When is a portfolio of small Qwen variants better than one large Qwen model?

## Framing the decision Qwen is unusual among open-weight families in offering a nearly continuous range: sub-1B edge models through mid-size dense checkpoints to large Mixture-of-Experts flagships, plus specialist lines for code, vision, audio and retrieval. That breadth means the question "which Qwen model do we run" is really "how many, and why" — a portfolio question of the same shape as choosing how many database engines an organisation supports. ## The case for a portfolio **Cost follows the model you actually invoke.** If 60% of your traffic is intent classification and routing, serving that on a flagship is pure waste — a small dense checkpoint answers it at a fraction of the compute. Traffic mixes in real products are usually far more skewed than teams expect, and the skew is where the savings live. **Latency budgets differ by surface.** An autocomplete box and a research assistant cannot share a model without one of them being wrong. A small model on the interactive path and a large one behind an async job is not a compromise; it is the correct design. **Modality is a hard gate.** Image-bearing requests need a VL checkpoint; audio needs the Omni/Audio line; retrieval needs an embedding model. These are not optional specialisations you can collapse into a generalist, so any product touching multiple modalities already runs a portfolio whether it planned to or not. **Blast radius shrinks.** Independent models mean a regression in the code path does not degrade the support chat. Isolation is a real reliability property, and it is the same argument that justifies separate service deployments. **Hardware fit improves.** Several small checkpoints can be placed on cheaper, more available accelerators and scaled independently against their own traffic curves, instead of one large deployment sized for the sum of unrelated peaks. ## The case for consolidation **Every model is a long-lived operational commitment.** Resident memory, warm capacity, a container image, a version pin, an evaluation suite, a rollback plan, a licence record, an on-call runbook. That cost recurs forever and is paid by people, not GPUs. Three models are not three times one model — coordination makes them worse than linear. **Evaluation debt compounds.** Each model needs its own regression set, refreshed as prompts change. Teams add specialists enthusiastically and then stop re-evaluating them; six months later nobody can say whether the Coder checkpoint still beats the generalist, and nobody dares remove it. **Utilisation collapses on small models.** A specialist that sees 2% of traffic still holds its weights in memory and still needs enough replicas for availability. At low volume, a shared larger model at high utilisation is genuinely cheaper than three idle small ones — the maths flips entirely on request rate. **Behavioural consistency is a product property.** Different checkpoints have different refusal behaviour, formatting habits, tone and tool-calling reliability. Users notice when the assistant changes personality across features, and every prompt technique you develop has to be re-validated per model. ## How to decide, concretely 1. **Measure the traffic mix first.** Requests per second and token volume per task class. Without this, every argument is anecdote. 2. **Apply the hard gates.** Modality requirements and hard latency budgets remove options before any cost discussion — they are not tradeable. 3. **Establish the baseline.** One general instruct checkpoint at a size that clears quality on the *hardest* task class. Measure per-task accuracy, p95 latency and cost. 4. **Propose a split only with numbers.** A specialist earns its place when the measured saving or quality gain over the baseline exceeds the ongoing cost of operating it — engineer time included, not just GPU hours. 5. **Attach a review date and an exit rule.** State up front what result would cause you to retire the specialist, and re-measure on a schedule. Portfolios rot because nothing ever obliges a removal. 6. **Cap the count deliberately.** Decide how many distinct model deployments the team can genuinely maintain and treat that as a budget, so adding one forces a conversation about removing another. ## Patterns that work well **Tiered routing.** A small model handles the bulk; a large one is escalated to by a classifier, an explicit user action, or a failed verification. This captures most of the cost saving of a portfolio with only two models to operate. **Offline versus online split.** Embedding and batch enrichment run on small models on cheap capacity; interactive traffic gets the good checkpoint. The workloads never contend, and the offline side can be preempted. **Modality edge only.** One general text model plus exactly one VL checkpoint on the image path — the minimum portfolio a multimodal product can have. ## Anti-patterns Adding a specialist because a benchmark table said it was better, without measuring on your own traffic. Running five checkpoints when three see negligible volume. Letting each team pick its own model, so the organisation discovers eleven deployments during a cost review. And the mirror-image failure: forcing one flagship onto every path, including classification, and treating the resulting bill as the unavoidable price of using LLMs. ## The honest summary The Qwen family is broad enough that both answers are defensible, which is exactly why an interviewer asks. The strong answer names the gates that are non-negotiable, insists on traffic measurement before architecture, prices the human cost of each additional model, and — most tellingly — specifies in advance what evidence would cause the portfolio to shrink again.