What does a 95% Bayesian credible interval of [1.9%, 2.5%] say about a conversion rate?
answer
- ask what the probability is attached to
- it is a summary of a distribution
- 95% of the posterior mass sits inside
- the claim is about the parameter itself
- conditional on prior, model and data
basics
~20 sIt says 95% of the posterior probability for the conversion rate falls between 1.9% and 2.5%. Given the model and prior you used, there is a 95% probability the rate lies in that range. The probability attaches to the parameter.
solid answer
~40 sA credible interval is a range that holds a stated share of the posterior distribution. So [1.9%, 2.5%] at the 95% level means: conditional on the data, the likelihood model and the prior, there is a 95% probability the true conversion rate lies between 1.9% and 2.5%. That direct reading is legitimate here because in the Bayesian setup the unknown rate has a probability distribution, and the interval is just a summary of it. Two caveats I would state out loud. First, the statement is conditional on the prior and the model, so a different prior gives a different interval and a different probability claim. Second, the probability sits on the parameter, not on the procedure - which is exactly the reading you are not allowed to give a frequentist confidence interval.
go deeper
Be ready to state the direct reading in one clean sentence: 95% posterior probability that the parameter is in the range, given the model and prior. Practise saying it without hedging.
Explain where the interval comes from - posterior mass between two endpoints - and why the interval is not unique until you pick a rule such as equal-tailed or shortest.
Show the judgment: name the prior and model in the same breath as the numbers, and flag when a skewed or boundary-hugging posterior makes a two-number summary misleading.
Own the reporting standard. Decide what a credible interval must be published with - prior, interval rule, posterior plot - so that readers across the organisation cannot quietly misread a conditional claim as an unconditional one.
## The object being summarised In Bayesian analysis the unknown quantity - here a conversion rate, call it `r` - is treated as a random variable with a probability distribution. Before seeing data you specify a **prior** distribution over `r`. You then observe data and combine the two through the likelihood, which is the probability of the observed data for each candidate value of `r`. The result is the **posterior** distribution: `p(r | data)` proportional to `p(data | r) * p(r)`. The posterior is a full distribution over every possible value of the rate, not a single number. A posterior is awkward to put in a report, so you summarise it. A **credible interval** at level 95% is any range `[L, U]` whose posterior probability is 0.95 - that is, the area under the posterior between `L` and `U` is 0.95. ## Reading the interval Because `r` genuinely has a distribution in this framework, the plain-English reading is the correct one: *given this model, this prior and this data, there is a 95% probability that the conversion rate lies between 1.9% and 2.5%*. There is no extra machinery to unpack, no hypothetical repetition of the experiment, no distinction between the interval you computed and some other interval you might have computed. That directness is the main practical selling point of the Bayesian summary, and interviewers ask this question specifically to see whether you state it cleanly rather than hedging it into something weaker. Equivalently: if you drew a very large number of values of `r` from the posterior, about 95% of those draws would fall between 1.9% and 2.5%. ## What the statement is conditional on The probability claim is not unconditional truth. It is conditional on three inputs: 1. **The data.** More data usually concentrates the posterior and narrows the interval. 2. **The likelihood model.** If you modelled visitors as independent Bernoulli trials with a constant rate and in reality the rate drifted over the week, the posterior is answering a question about a model that does not describe the world. 3. **The prior.** With plenty of data the prior barely matters; with little data it can dominate, and the interval width will move noticeably when you change it. This is why a good report states the prior next to the interval. "95% credible interval [1.9%, 2.5%] under a uniform prior on the rate" is a complete claim; the bare interval is not. ## Which 95% interval Infinitely many ranges hold 95% of the posterior mass, so the interval is not unique until you fix a rule. The two standard rules are the **equal-tailed** interval, which cuts 2.5% of the posterior off each tail, and the **highest posterior density** interval, which takes the shortest range holding 95%. On a roughly symmetric posterior they are almost identical; on a strongly skewed one they are not. Whatever you use, say which one it is - the probability statement is the same for both, but the endpoints are not. ## The contrast worth one sentence A frequentist confidence interval attaches its probability to the *procedure* that produced the interval, not to the parameter, so "there is a 95% probability the rate is in this interval" is not a licensed reading there. In the Bayesian case it is licensed, because the parameter is the thing carrying the distribution. Interviewers often set the trap in reverse: they hand you a credible interval and see whether you refuse the direct reading out of reflex. Refusing it is as wrong as giving it for the frequentist interval. ## Practical reporting A few habits that make the number defensible: - Report the interval together with a point summary (posterior mean or median) so the reader sees both centre and spread. - Name the prior and the interval rule in the same sentence as the numbers. - Show the posterior itself when it is skewed or piles up near a boundary; a two-number summary can hide a lot of shape. - Do not treat the endpoints as hard limits. Values just outside 1.9% and 2.5% still carry posterior probability - 5% of it, split between the two sides - they are simply less plausible than the ones inside. ## Common failure in interview The answer that loses points is the one that says "95% of users convert at a rate between 1.9% and 2.5%", which confuses a statement about a single unknown parameter with a statement about a population of individuals. The interval describes uncertainty about one number, the site-wide rate, not variation across people.
- Besides the observed data, what determines where the endpoints land?The prior and the likelihood model. The posterior is proportional to likelihood times prior, so both feed the interval. With a small sample the prior can move the endpoints substantially; with a large sample it usually washes out. That is why the prior belongs in the report alongside the numbers.
- What changes if you report a 99% credible interval instead of 95%?The interval gets wider, because it has to hold more posterior probability - 99% of the mass instead of 95%. The centre stays put. Nothing about the data or the posterior changes; you have only chosen a more demanding coverage level for the summary.
- Can two analysts with the same data report different 95% credible intervals?Yes, for two legitimate reasons. They may have used different priors or different likelihood models, giving different posteriors. Or they may have used the same posterior but different interval rules - equal-tailed versus shortest - which pick different 95% slices of the same distribution.
It is a weather forecast for one unknown number: 95% probability the rate sits in this band, given everything the model was told.
saying these in an interview costs you the question
- States the probability without naming the prior or model
- Reads it as 95% of users converting inside the band
- Treats the endpoints as values the rate cannot exceed
- Puts the probability on the estimate rather than the parameter
- Assumes the 95% interval is unique for a given posterior