What makes a Beta prior conjugate to a binomial likelihood in Bayesian updating?

## The general idea Bayes' rule says the posterior is proportional to the prior times the likelihood: `posterior(theta | data) = prior(theta) * L(theta; data) / evidence` The denominator, the evidence, is the integral of `prior(theta) * L(theta; data)` over all values of theta. For most models that integral has no closed form, which is why Bayesian computation is often numerical. A prior family is called **conjugate** to a likelihood when the product `prior * likelihood`, viewed as a function of theta, has the same algebraic shape as the prior. When that happens you can read the posterior parameters straight off the exponents and never touch the integral, because you already know the normaliser of that family. Conjugacy is a property of a *pair*: a family of priors together with a likelihood. It makes no sense to call a prior conjugate on its own. ## Why Beta and binomial fit together A `Beta(a, b)` distribution lives on the interval `(0, 1)`, which is exactly where a probability parameter lives. Its density is `f(theta) = theta^(a-1) * (1-theta)^(b-1) / B(a, b)` where `B(a, b)` is a constant that does not depend on theta. Its mean is `a / (a + b)`, and when `a > 1` and `b > 1` its mode is `(a - 1) / (a + b - 2)`. Observing `s` successes in `n` independent trials with success probability theta gives a likelihood proportional to `theta^s * (1-theta)^(n-s)` The binomial coefficient is dropped because it does not involve theta, and anything free of theta is absorbed into the normaliser. Multiply the two: `theta^(a-1) * (1-theta)^(b-1) * theta^s * (1-theta)^(n-s) = theta^(a+s-1) * (1-theta)^(b+n-s-1)` That is the kernel of a `Beta(a + s, b + n - s)`. The update rule is therefore: **add the successes to the first parameter and the failures to the second**. A common slip is adding `n` to the second parameter instead of `n - s`. ## The pseudo-count reading Because the update just adds counts, the prior parameters behave like counts you pretend to have already seen. A `Beta(3, 7)` prior on a conversion rate acts like 3 prior successes and 7 prior failures — a prior mean of 0.3 carrying about 10 observations of weight. `Beta(30, 70)` has the same mean but ten times the weight and moves ten times less when the data arrives. The posterior mean makes the weighting explicit: `(a + s) / (a + b + n)` which can be rewritten as `[(a + b) / (a + b + n)] * [a / (a + b)] + [n / (a + b + n)] * [s / n]` an average of the prior mean and the observed proportion, weighted by prior strength `a + b` against sample size `n`. As `n` grows, the second term dominates and the posterior mean converges on the observed proportion. That is the precise sense in which data eventually outweighs a fixed prior. ## Other standard pairs The same pattern recurs across the exponential family: - **Gamma prior, Poisson likelihood** for a rate: the prior contributes pseudo-events over pseudo-time-units. - **Normal prior, Normal likelihood with known variance** for a mean: precisions add and the posterior mean is a precision-weighted average. - **Dirichlet prior, multinomial likelihood** for a probability vector: the multivariate generalisation of Beta-binomial, updated by adding category counts. - **Gamma prior, exponential likelihood** for a rate: add the number of observations to the shape and the sum of the waiting times to the rate parameter. Every likelihood in the exponential family has a conjugate prior family; the recipe is always "prior parameters accumulate the sufficient statistics of the data". ## What conjugacy does and does not buy you It buys: closed-form posteriors, exact answers with no sampler and no convergence diagnostics, constant-size state (you carry two numbers, not a dataset), and a posterior that is itself a valid prior for the next batch. That combination is why conjugate updates are the natural fit for online systems that maintain a rate estimate per item. It does not buy correctness. Conjugate families are narrow and unimodal in the usual parametrisations, so if your genuine belief is "this rate is either near zero or near one half", no single Beta expresses it and forcing one distorts the analysis. Conjugacy also evaporates the moment the model gains structure the family cannot absorb — covariates, unknown dispersion, measurement-error layers. The honest framing in an interview is: conjugacy is a computational convenience that happens to be available for a handful of very common models, and choosing a prior because it is convenient rather than because it describes your belief is a decision you should be able to defend.