When is the convenience of a conjugate prior not worth the constraint it puts on your model?

## What conjugacy is actually buying Before deciding when to give it up, be precise about the value: - **Exactness.** The posterior is a formula, not an approximation. Nothing to diagnose, no convergence to argue about, no variability between runs. - **Constant state.** Two or three numbers per entity replace the event history. For a system tracking a rate per item across a large catalogue, that is the difference between a small table and a warehouse scan. - **Cost per update.** An update is a few additions, so it can live on the request path. - **Auditability.** "We added 8 successes and 92 failures to the prior" is a sentence a reviewer, a regulator or a sceptical stakeholder can follow. Explaining a sampler to the same audience is harder. - **Composability.** The posterior is a valid prior, so batch and streaming paths reconcile exactly. That is a strong package, and dismissing conjugate models as toys is a common overcorrection by candidates who have recently learned numerical inference. ## Trigger one: the family cannot hold the belief Conjugate families are small. A Beta is unimodal for parameters above one, so a genuinely bimodal belief — "either this feature is broken and the rate is near zero, or it works and the rate is near the usual few percent" — has no single Beta representation. Choosing the closest Beta does not merely lose detail; it puts mass where you believe there is none, and every downstream summary inherits that. Similar failures: a hard bound you know exists but the family smears past, an asymmetric belief the family cannot bend to, or a prior that has to encode a relationship between two parameters when the conjugate form treats them as independent. The partial escape is a **finite mixture of conjugate priors**. A mixture of Betas updated against binomial data yields a mixture of Betas — the components update independently and the mixture weights get reweighted by how well each component predicted the data. You keep closed form and buy multimodality, at the cost of carrying a handful of components instead of two numbers. Reaching for that before abandoning tractability entirely is a good instinct to show. ## Trigger two: the model needs structure the family has no form for This is the harder wall, and it is usually where real work lands. Conjugate updating exists for a short list of likelihoods with no explanatory structure. As soon as you want - the rate to depend on covariates, - extra variance beyond what the count model allows, - a measurement-error or missing-data layer between the parameter and what you observed, - or several parameters that must be inferred jointly, there is generally no conjugate prior to be had, and no amount of cleverness recovers one. At that point the choice is not "conjugate or numerical" but "the wrong model with a closed form, or the right model without one". ## The actual decision Frame it as a comparison of two costs. **Cost of staying conjugate:** the modelling error from the constraint. Quantify it rather than asserting it — how far off is the posterior mean, how badly are the intervals miscalibrated, does the decision the number feeds actually change? Often the answer is that it does not, and the constraint is free. **Cost of leaving:** engineering and operations. Numerical posterior approximation means a sampler or an optimiser in the serving path or a batch job, convergence checks that someone must own at 3am, run-to-run variability that complicates testing, and a model fewer people on the team can debug. None of these is prohibitive; all of them are real, recurring costs. The scale and the stakes decide. A per-item rate updated millions of times a day, feeding a ranking that tolerates small errors: stay conjugate. A single high-stakes estimate computed weekly, feeding a decision worth real money: the operational cost is trivial and the fidelity is worth everything. ## Hybrid positions worth naming Seniority shows in refusing the binary. - **Conjugate core, richer periphery.** Keep the closed-form update for the high-volume component and model the complicated part separately. - **Conjugate as the fast path.** Serve from the closed-form posterior, and recompute periodically with the richer model to detect when the approximation has drifted. - **Conjugate as the baseline.** Ship it first, measure the decision quality, and let the richer model earn its complexity against a real number. ## The failure modes on both sides Staying too long looks like a prior nobody believes, quietly chosen because it made the algebra close, with the belief bent to fit the family and never revisited. Leaving too early looks like a complex model where a counter would have done, whose maintenance burden outlives the person who built it. ## What to say Name both triggers — expressiveness and structure — give the mixture-of-conjugates middle path, and then make the decision a cost comparison with the volume and the stakes as the deciding variables. The answer an interviewer is listening for is not "conjugate is outdated" but "conjugacy is a constraint I accept deliberately, and here is the evidence I would gather before paying to escape it".