How would you run Thompson sampling on a news feed where articles arrive and expire daily?

## The rule needs no modification Start with what does *not* break. Thompson sampling defines a per-request procedure: draw one number from each live arm's posterior, serve the argmax. Nothing in that procedure assumes a fixed arm set. Publish a new article and it joins the draw with its prior; expire an old one and it drops out. No special-casing, no forced-exploration branch, no cold-start rule bolted on top — a new arm gets traffic precisely because its posterior is wide and therefore plausibly best. That property is a genuine reason to reach for this rule in a churning environment. Everything hard is around the edges. ## Failure one: exploration swamps exploitation With K live arms whose posteriors are all near the prior, each wins about one draw in K. If ten articles are live, that is fine — 10% of traffic goes to any given newcomer and the pool resolves within an hour. If five hundred are live and a hundred turn over daily, the arithmetic collapses: nearly all traffic is spent on arms nobody has any evidence about, and the well-performing known arms are crowded out. The sampler is behaving correctly and the outcome is terrible. The fix is upstream of the sampler. Bound the candidate pool: retrieve a small shortlist by some cheap heuristic — recency, section, editorial priority, a coarse model score — and run the sampler only over that shortlist. The bandit is a *ranker of a handful of finalists*, not a mechanism for exploring an unbounded catalogue. An interviewer is often probing exactly for the recognition that arm count, not the rule, is the binding constraint. ## Failure two: the flat prior is a lie Giving a brand-new article a uniform prior says you believe its click rate could plausibly be 80%. You do not believe that. You have thousands of past articles and you know their rates cluster tightly — say most of them between 1% and 6%. Encoding that is the single highest-leverage change. Fit the prior for a new arm from the historical distribution of comparable articles, matched on the attributes you actually have at publication time: section, source, position, time of day. The new arm then starts centred near the population mean with a width equal to the genuine between-article variation, so it is explored in proportion to how much articles *really* differ — not in proportion to total ignorance. Two direct benefits: exploration cost per new arm drops sharply, and the arm-count problem above becomes much less acute, because a newcomer no longer routinely draws absurdly high values. The risk to name is over-confidence. If the fitted prior is too tight, a genuinely exceptional article never wins draws and is never discovered — the prior has to carry the real spread across articles, not the average article's uncertainty. ## Failure three: rewards arrive after decisions An impression is served now; the click, if it comes, lands seconds or minutes later, and a downstream conversion may take longer still. If the posterior is only updated when rewards land, then for the duration of the delay the sampler sees an arm that has served heavily but shows no evidence, keeps a wide posterior, and keeps serving it. At high throughput that means thousands of impressions committed to an arm on the strength of nothing. Two standard remedies. Provisionally record the impression as a non-reward at serve time and correct it when the click lands — this makes the sampler pessimistic during the delay window, which is the safe direction. Or batch: freeze the allocation, serve for a fixed interval longer than the reward delay, then update everything at once. Batching is coarser but far simpler to reason about and to operate. ## Failure four: the horizon per arm is tiny Adaptive allocation earns its keep by exploiting for a long time after a short exploration phase. An article that expires in 24 hours may accumulate only a few thousand impressions, which for a one-point gap is nowhere near enough for posteriors to separate. Per-arm, the sampler spends most of the article's life still exploring, and the regret saving is small. The answer is to stop treating each article as an isolated arm. Share strength across arms: pool at the level of section, source or template, so evidence from one article informs the prior of the next. The individual arm's short life then rides on a slowly-learned population structure rather than starting from scratch every time. ## Operational judgment to voice - **Retire cleanly.** An expired article leaves the draw set immediately, but keep its accumulated posterior as an input to the population prior — that is where its long-term value lies. - **Bound allocation velocity.** A rule that can commit most of the feed to one article within minutes turns a tracking glitch into a visible incident. Cap how fast any arm's share may grow. - **Monitor served shares, not just rewards.** The diagnostic that catches all four failures above is the distribution of traffic across arms over time: exploration swamping shows as a flat share curve, over-confident priors show as newcomers never appearing, and delayed rewards show as an arm's share spiking before any evidence exists. - **Know when the answer is no.** If the pool is enormous, lifetimes are short and per-arm traffic is thin, per-item adaptive allocation is the wrong tool and the leverage lies in the retrieval and prior-fitting layers instead. ## The compact answer The rule survives churn untouched; what needs engineering is the number of live arms, the prior a new arm starts from, the delay between serving and learning, and the fact that a one-day article never lives long enough to be exploited.