Why does Thompson sampling keep serving an arm with only 1 success in 2 trials?

## The observation Run a Thompson sampler over a few arms and watch the served shares. An arm that has been tried twice keeps picking up requests, seemingly out of proportion to anything it has demonstrated. Meanwhile an arm with a thousand trials and a mediocre rate gets almost nothing. Both behaviours come from the same source: the width of the posterior. ## Width, quantitatively The posterior is a distribution over the arm's unknown true rate, and its width shrinks roughly with the square root of the number of trials. Two concrete cases, both with a flat starting prior: - **1 success in 2 trials.** The posterior is centred at 0.5 with a standard deviation of about 0.22. Draws from it routinely land anywhere from 0.15 to 0.85. This arm knows essentially nothing about itself. - **500 successes in 1,000 trials.** The posterior is centred at 0.5 with a standard deviation of about 0.016 — roughly fourteen times narrower. Draws stay inside about 0.45 to 0.55. Both arms have the same point estimate. They behave completely differently under sampling, because the rule compares draws, not centres. ## What that does to allocation Suppose the leader's posterior is concentrated near 0.55. The 500/1,000 arm will practically never draw above it — its whole distribution sits below. The 1-of-2 arm draws above 0.55 a large fraction of the time, so it wins requests regularly. From the outside this looks like the sampler favouring an unproven arm; in fact it is the sampler correctly saying "I cannot yet rule out that this arm is the best one, and the only way to find out is to serve it". The symmetric case is instructive. If two arms have posteriors centred at the same value but very different widths, neither is favoured — by symmetry each wins about half the draws. Width does not by itself attract traffic; width attracts traffic *when the arm is behind*, because width is what puts posterior mass above the leader. ## Self-annealing The important consequence is that exploration decays on its own. Every request the uncertain arm wins produces an observation, which narrows the posterior, which reduces the chance it wins the next draw. If the arm really is mediocre, a few dozen trials shrink it out of contention. If the arm really is excellent, the same trials pull its posterior up and it takes over. Either way, no schedule is written by hand and no parameter needs decaying — the feedback loop between traffic and certainty does the annealing. This is what people mean when they say Thompson sampling has no exploration knob. The knob exists, but it is the prior: a strong, confident prior on an arm makes its posterior narrow before any data arrives, and it will be under-explored. A weak prior leaves it wide and it will be explored generously. Prior specification is the real lever. ## Where the intuition breaks **Confident priors starve arms.** If you initialise a new arm with a prior equivalent to hundreds of pseudo-observations at a low rate, its posterior is narrow from the start, it never wins draws, and it never gets the data to correct itself. The self-correcting loop requires that the arm's posterior overlap the leader's. **Delayed rewards distort width.** The posterior only narrows when rewards land. If an arm has served thousands of impressions whose outcomes have not yet arrived, its posterior is still wide and it keeps winning draws, so it gets served far more than intended before the evidence catches up. **Width is not goodness.** A weak candidate answer is "the sampler prefers uncertain arms". It does not prefer them; it gives them a hearing proportional to how plausible it is that they are best. An uncertain arm whose posterior lies entirely below the leader's — possible if the prior was informative — gets nothing. ## The interview-ready summary Thompson sampling allocates traffic to an arm in proportion to the posterior probability that the arm is the best one. A barely-tested arm has a wide posterior, so that probability is meaningfully above zero and it earns traffic. A heavily-tested but clearly worse arm has a narrow posterior sitting below the leader, so that probability is near zero and it gets squeezed out. Exploration is bought with uncertainty and paid off with data.