Bayesian Statistics

The frequency reading defines probability as the proportion of times an outcome occurs in repeatable trials. The belief reading defines it as a numeric degree of confidence, so it can also score one-off events such as a single rocket launch.

Beta(1 + 8, 1 + 92), that is Beta(9, 93). The prior contributes one pseudo-success and one pseudo-failure, so the posterior mean is 9/102, about 0.088, slightly above the raw rate of 0.08.

It says 95% of the posterior probability for the conversion rate falls between 1.9% and 2.5%. Given the model and prior you used, there is a 95% probability the rate lies in that range. The probability attaches to the parameter.

Probability fixes the coin's bias and varies the data; likelihood fixes the observed data and varies the bias. After 7 heads in 10 flips the likelihood is a curve over p that is not a probability distribution.

Maximum likelihood picks the parameter value that makes the observed data most probable. MAP picks the value that maximises the posterior, which is the likelihood multiplied by a prior. When the prior is flat, the two estimates coincide.

MCMC produces a stream of parameter draws that visit each region in proportion to posterior probability. You never compute the normalising integral: every posterior summary, such as a mean or a tail probability, becomes an average over those draws.

The posterior predictive distribution is the distribution of a future observation after averaging the likelihood over every parameter value the posterior still finds plausible. It carries sampling noise plus parameter uncertainty, instead of conditioning on a single estimate.

An informative prior encodes real outside knowledge and moves the posterior by itself. A weakly informative prior only rules out implausible values, letting the data dominate. A flat prior spreads density evenly and is not genuinely neutral.

Sample a large set of values from each arm's posterior, pair the draws index by index, and take the fraction of pairs in which B's draw exceeds A's. That fraction is a Monte Carlo estimate of the probability of superiority.

Frequentists treat the unknown parameter as a fixed constant and put all the randomness in the data. Bayesians give the parameter a probability distribution that describes how uncertain they are about its value, and condition on the data actually observed.

A Beta density and a binomial likelihood are both powers of theta and 1 minus theta, so their product is again a Beta: a Beta(a, b) prior with s successes in n trials gives Beta(a + s, b + n - s).

Because MCMC draws are dependent. When each draw is nearly a repeat of the previous one, 10,000 correlated draws carry about as much information as 200 independent ones, and effective sample size reports that count rather than the stored count.

R-hat compares the variation between several MCMC chains with the variation within each chain. If the chains have converged to the same distribution the two agree and R-hat sits near 1; values above roughly 1.01 mean the chains still disagree.

An equal-tailed interval cuts equal posterior probability off each tail: at 95% it runs from the 2.5th to the 97.5th percentile. A highest posterior density interval is the shortest range holding 95%, so on a skewed posterior it is shorter.

It means the posterior density at each parameter value is prior times likelihood divided by a single constant, the evidence. That constant does not depend on the parameter, so the product alone determines the posterior's shape.

The acceptance rule uses the target only as a ratio of its values at the proposed and current points. Both carry the same constant factor, so it cancels and the sampler needs nothing beyond likelihood times prior.

A predictive interval must cover a single random observation, so it carries the model's sampling noise on top of the uncertainty about the parameter. A parameter interval carries only the second piece, and shrinks toward zero width as data accumulate.

Thompson sampling draws one random value from each arm's posterior over its reward rate, then serves the arm whose draw came out largest. Every request repeats the draw, so allocation follows the posteriors on its own.

Maximising the evidence lower bound (ELBO) picks the distribution in a chosen tractable family that is closest to the true posterior. The bound's gap to the log evidence is exactly a KL divergence, so raising the ELBO shrinks that gap.

Early MCMC draws still reflect where the chain was started rather than the posterior, and in adaptive samplers the tuning is still changing. Those draws are dropped so the summary uses only draws from the stationary distribution.

As a precision-weighted average, where precision is one over variance. Precisions add, so the posterior is always more precise than the prior, and its mean lies between the prior mean and the sample mean, closer to whichever is more precise.

Inside a single model the marginal likelihood is a constant that only rescales the posterior, so you can ignore it. You must compute it to compare models, since a Bayes factor is a ratio of two marginal likelihoods.

Taking logs turns the posterior into log-likelihood plus log-prior. A zero-mean Gaussian prior contributes minus the sum of squared coefficients divided by twice the prior variance, so maximising it is least squares with an L2 penalty attached.

A Gibbs sampler draws each block from its exact full conditional, its distribution given the current values of all other parameters and the data. Because the proposal is already exact, the Metropolis acceptance probability equals one.

A prior predictive check simulates fake datasets from the prior and the likelihood, then asks whether those datasets are physically sensible. It catches priors that imply absurd observables, such as four-metre human heights or negative revenue, before any fitting happens.

A density picks up a Jacobian factor under a change of variables. A uniform prior on p in [0,1] becomes a bell-shaped density on log-odds peaked at zero, favouring p near 0.5. No prior is uninformative on every scale.

Two trials leave a very wide posterior, so a draw from that arm lands above the leader's draw often enough to win some requests. Uncertainty, not a tuned exploration parameter, is what buys the arm its traffic.

A mean-field family factorises the approximation into independent pieces, so it cannot represent correlation between parameters. On a correlated posterior it fits inside the narrow direction of the ridge rather than along it, producing spreads that are too small.

Expected loss of shipping B is the posterior average of how much worse B is than A, counting zero where B is better. It is in metric units, so you compare it to a tolerance.