Self-Reinforcing Cycles

## The training set is a photograph of the model's own choices A priority-inbox ranker scores each arriving message and promotes a small share of them into a prominent view; the rest land in a secondary view that recipients scroll less often. A month later the refresh is assembled: a message the recipient opened becomes a positive, a message they did not becomes a negative. That looks like a sample of recipient preference. It is a sample of preference **filtered through last month's placement decision**. Opening a message requires seeing it, and seeing it was decided by the ranker. So the assembled training set answers a narrower question than the one the model is asked at serving time: not *would this recipient want this message*, but *did this recipient open this message, given that the previous ranker chose to show it prominently*. ## Why the second refresh is worse than the first The loop turns once per refresh, and it compounds: 1. The ranker assigns a low score to a sender class — shipping notifications, say — for whatever reason it had in month one: thin history, an unlucky feature value, a scoring quirk. 2. Mail from that class is demoted, so far fewer recipients look at it. 3. Opens from the class collapse — not because interest fell, but because the placement that produces opens was withdrawn. 4. The refresh assembles month two's data and finds the class represented almost entirely by unopened messages, labelled negative. 5. The refit learns a lower score for the class than month one's model held, and step 2 runs harder. The distinction worth stating in a design review is between a **static blind spot** and a **tightening one**. A model fitted once on censored data has a fixed hole. A model that produces the log its own successor is fitted on has a hole that widens every cycle, because each refresh's evidence was generated by the previous refresh's beliefs. Nothing inside the pipeline pushes back on it. ## The mirror on the promoted side The same mechanism runs with the sign flipped. A sender class promoted in month one accumulates exposures, therefore opens, therefore positives, therefore a higher score at the next refit, therefore more promotions. Two sender classes of genuinely equal interest can end up several refreshes later with a wide score gap that records which one was promoted first rather than which one the recipient prefers — the accumulation the **Matthew effect** names. This side is easier to miss because the headline metric looks healthy: the promoted view's open rate is high, precisely because it is measured on the mail the ranker was already most sure about. ## What moves and what does not | Quantity | Across successive refreshes | Why | |---|---|---| | recipient interest in the demoted class | stable, in the scenario as described | nothing about the recipient changed | | exposures logged for that class | falls sharply after the first demotion | placement decides exposure | | positives in the next training set | falls with exposure, approaching zero | an open needs an exposure first | | the class's learned score | falls further at each refit | the class is now almost all negatives | | model confidence on that class | tends to rise | it is scored on data it selected | | the offline evaluation number | can rise while live quality is flat | the held-out set is censored the same way | That last row is the trap. If the evaluation split is carved out of the same logged traffic, it inherits the same censoring, so the refresh reports an improvement on exactly the population the model has stopped doubting. ## What does not break the cycle - **More data.** Ten times the volume of the same censored log is ten times as much evidence that demoted mail goes unopened. - **A longer training window.** Older months were produced by earlier models in the same loop; the history is censored too, just less tightly. - **A larger model or richer features.** Capacity fits the censored distribution more sharply, which usually makes the demotion crisper, not softer. - **A faster refresh.** Turning the loop more often tightens it sooner; cadence changes the speed, not the direction. - **Freezing the ranker for a while.** Freezing stops further hardening, but the log still contains only what the frozen model chose to show, so the training set does not repair itself. ## What does Evidence the current ranker did not select. In practice that is a small randomised share of the promotion slots, filled without reference to the score, so some mail from suppressed classes is seen and its opens are recorded. Alongside it, recording the selection probability at decision time lets the next refresh re-weight classes that were under-exposed rather than absent. A third, weaker signal is a rescue action — a recipient digging a message out of the secondary view — which produces an occasional positive from a class the ranker had written off, though far too few of them to re-seed a training set on their own.