Bayesian Statistics

A region of practical equivalence is a band of differences you would call 'no real difference', fixed before analysis. If the posterior for the lift sits entirely inside it, the arms are equivalent and you choose on cost.

At 40 sessions the raw 7.5% rate is mostly noise: one more conversion would read 10%. A prior built from comparable past features supplies the information the data lacks and pulls the estimate toward plausible values.

Because the posterior becomes the next prior and the update only adds sufficient statistics. For exchangeable data the likelihood factorises and multiplication commutes, so any batching or ordering accumulates the same success and failure counts and lands on identical parameters.

Treat the fit as biased, not noisy. Divergences near a near-zero scale parameter signal the funnel geometry a centred hierarchical parameterisation creates; the fix is a non-centred parameterisation, then a smaller step size.

Report the prior explicitly and quantify its weight - a Beta prior contributes roughly its two parameters as pseudo-observations - then publish the interval under a ladder of priors. A narrow interval from a strong prior is an assumption, not evidence.

With two separated modes the posterior mean lands in the trough between them, a value the posterior itself calls unlikely. The single number also hides that two competing explanations are in play.

A skewed posterior has three different point summaries. Under right skew the mode is the smallest of the three, and it optimises an all-or-nothing loss that matches almost no real decision.

The proposal steps are far too large, so almost every candidate lands in a low-density region and is rejected and the chain sits still for long stretches. Shrink the proposal scale until acceptance rises to roughly a quarter.

The fitted model cannot generate the excess zeros the real data contain, so it is misspecified for that feature. Quantify the gap with the proportion of zeros as a test statistic, then extend the model to produce structural zeros.

It treats the eight group effects as draws from a common distribution whose mean and spread are learned from the data. Each estimate is then pulled toward the overall mean, the noisiest groups moving furthest.

A long-running Thompson sampler locks in because its posteriors become extremely tight and the changed arm barely gets served. Make it forget on purpose: discount old observations each round, or keep only a sliding window of recent ones.

A fixed 50/50 split's cumulative regret grows linearly with traffic: 50,000 impressions parked on a 4% arm instead of a 5% arm costs about 500 conversions. Thompson sampling's grows sublinearly, roughly like the logarithm of the horizon.

KL(q||p) takes expectations under the approximation you control, so it is computable up to a constant; the reverse direction needs expectations under the unknown posterior. The price is mode-seeking behaviour that ignores parts of a multimodal target.

The threshold should encode the cost of being wrong, not a borrowed convention. Set it tighter for expensive, hard-to-reverse changes and looser for cheap reversible ones, pair it with a magnitude requirement, and fix it before the experiment runs.

Decide by the shape of the decisions, not by taste. Bayesian reporting pays off when data per decision is thin, credible prior information exists, and stakeholders need a probability about the claim itself. Frequentist reporting suits high-volume standardised readouts.

When the family cannot express the belief or the structure the problem has. Closed form buys exact, constant-memory updates worth keeping at high throughput, but bending a bimodal belief or a covariate-driven model into a convenient family is a modelling error.

Report the flip as the finding: if the decision changes across priors reasonable people hold, the data are not decisive. Pre-specify the prior set, quantify the tipping point, and decide on the cost of being wrong.

Decide by which error you can afford. Variational inference is fast and scales, but its approximation error is biased and unmeasured; sampling is asymptotically exact but slow. Match the choice to how sensitive the downstream decision is to uncertainty.

Each observation multiplies more likelihood into the posterior, while the prior enters only once. With enough data the likelihood swamps it, so both posteriors concentrate on the same value and take the same approximately normal shape — the Bernstein-von Mises result.

Events add to the shape, exposure to the rate parameter: in shape-and-rate form, Gamma(alpha, beta) with y tickets over n days becomes Gamma(alpha + y, beta + n). The prior reads as alpha pseudo-events over beta pseudo-days.

No. For a fixed number of iterations, keeping every tenth draw throws away information, so estimates are no better and usually slightly noisier than using all draws. Thinning is justified by storage or downstream cost, not accuracy.

Jeffreys' rule sets the prior proportional to the square root of the Fisher information. For a binomial proportion that gives the Beta(1/2, 1/2) density. The motivation is invariance under reparameterisation, not neutrality.

It expands the log posterior to second order around its peak. The linear term vanishes there, leaving a quadratic, which is the log of a Gaussian centred at the peak whose covariance is the inverse of the negative second-derivative matrix.

The posterior itself needs no correction for how often you look: it summarises the data in hand at any moment. But a rule that stops the first time the probability crosses a bar still selects lucky data and overstates the lift.

Two likelihood functions that differ only by a factor free of the parameter give the same posterior, because that factor is absorbed by the normalising constant. With the same prior the two experiments' posteriors coincide exactly.

The posterior is a probability density, so changing variables multiplies it by a Jacobian that reshapes the curve and can move its peak. The likelihood is not a density over the parameter, gets no Jacobian, so its maximiser transforms along.

Momentum lets a proposal travel a long way while following the posterior's shape, so moves are coherent strides rather than a random walk. Distant proposals still get accepted, which is why exploration holds up as the number of parameters grows.

Make equal-tailed the default, because it is reproducible, survives rescaling and is always one interval. Require the highest-density version, plus the posterior plot, in named exception cases: strongly skewed posteriors, posteriors piled against a boundary, and multimodal ones.

Choose statistics that matter for the decision the model supports and that the fitting did not already force to match. A statistic the fit targets, such as the sample mean, passes regardless of how wrong the model is.

Thompson sampling handles arms coming and going without modification: each request draws over whatever arms are live, and a new arm's wide posterior earns it exploration. The real problems are cold-start priors, delayed clicks, and exploration swamping exploitation.