A news feed's ranked top ten carries at most two stories per outlet - which post-scoring rule produced that slate?
answer
- relevance alone did not build this page
- a pass after scoring, before the response
- the only stage that sees the whole slate
- walk the order, skip a capped outlet
- selection changes, the score does not
basics
~20 sA per-outlet slot quota in the list-editing pass that runs after scoring. The ranker orders candidates by relevance score; the quota then walks that order and skips any outlet that already holds two slots on the page.
solid answer
~40 sThat slate was not produced by relevance alone. A ranked feed ends with a **list-editing pass** - also called re-ranking or the policy layer - that takes the scored candidates and decides which of them actually occupy slots. A per-outlet quota is one of its edits: walk the candidates in descending relevance score, place each one unless its outlet already holds two slots, and defer the rest to a later slot or the next page. The quota never rewrites the relevance score and never asks the ranking model anything; it only changes selection. It can do this because it is the only stage that sees the slate as a set - the ranking stage scores one candidate at a time and has no idea what sits in slot three.
go deeper
Recall that a ranked feed ends with a pass that edits the list after scoring, and that outlet caps, de-duplication and takedowns all live there rather than in the model.
Explain the greedy walk: candidates in score order, each checked against the slate built so far, blocked ones deferred rather than deleted, and the shortlist over-fetched so the page does not come up short.
Show why this cannot move into the ranker - pointwise scoring has no view of the slate, rules change faster than retrains, and a legal constraint has to be a guarantee rather than a learned tendency.
Frame the layer as a cost centre: it trades measured relevance for constraints the score cannot express, and that trade is only defensible if each rule has an owner and a measured effect.
## The stage that produced the slate A ranked feed is not one stage, it is a funnel. Candidate retrieval pulls a few hundred or few thousand articles cheaply. A heavier **ranking stage** attaches a `relevance` score to each survivor, one candidate at a time, given the user and the article. Then a final pass decides which of those scored candidates actually occupy slots 1..N on the page. That last pass goes by several names - re-ranking, list editing, the policy or rule layer - and a per-outlet cap of two lives there. It has to live there, and the reason is structural: **it is the only stage that sees the slate as a set.** The ranking stage produces a number for one article at a time. It cannot know that the article it is scoring would be the third from the same outlet, because it has not been told what is already placed. ## Walking the scored order The mechanics are a greedy walk, and they are simpler than people expect: 1. Take the candidates in descending relevance score. 2. For each one, evaluate the rules against the slate built so far - here, how many slots this outlet already holds. 3. If no rule blocks it, append it to the slate. If a quota blocks it, **defer** it: leave it in the pool so it can fill a later slot or the next page. 4. Stop when the slate is full or the pool is exhausted. Nothing in that walk touches the score. The candidate that was skipped is still worth exactly what the ranker said it was worth; it simply did not get this slot. ## The other edits that share this pass The quota is one member of a family, and design rounds expect you to name the family: | edit | what it reads | what it changes | typical trigger | |---|---|---|---| | de-duplication | similarity between candidates | which candidate represents a cluster | forty outlets filing on one event | | slot quota | outlet or section of placed items | selection order | one outlet dominating the page | | eligibility filter | a rule list keyed by item or attribute | removes candidates outright | licence, region, paywall state | | suppression | a takedown list on the serving path | removes one specific item | a retracted story | | must-carry | an editorial pin | forces a slot | a major breaking story | | boost | an attribute such as article age | the relevance score itself | a fast-moving news cycle | Five of those six change **selection** and leave the score alone. The boost is the exception - it is the one edit in this layer that rewrites the number the ranker produced, which is why it is the one that has to be bounded and logged most carefully. ## Why the ranking model does not do this itself A reasonable junior answer is "train the model to dislike repetition." It does not work, for four separate reasons: - The ranking model is asked for a score given a user and an article. To penalise repetition it would have to be given the rest of the slate as input, which is a different and far more expensive ranker. - Rules change on a human timescale. An outlet cap gets loosened during an election week and tightened afterwards; a model retrains on a cadence measured in hours or days. - Some of these rules are **guarantees**, not tendencies. A licence restriction that holds 98% of the time is a broken licence restriction. A learned preference cannot promise anything. - Attribution dies. When someone asks why their story is not showing, a rule layer can answer with the rule id; a score cannot. ## What the layer costs Editing the list after scoring is not free, and an interviewer will usually ask about the cost: - **Measured relevance falls by construction.** You are, deliberately, not showing the highest-scoring set. The layer is worth its cost because the thing it protects - not showing ten versions of one story, not showing a retracted one - is not in the score at all. - **The slate can come up short.** If the rules remove more candidates than you expected, you need more candidates than slots. Over-fetching the shortlist is what pays for that. - **Offline replay does not see it.** An offline evaluation that replays the ranker's scores measures the unedited order, so the layer's effect shows up only in what was actually served. - **The layer needs its own logging.** Recording which rule moved or removed which candidate is the only way to debug a slate later.
- What happens to a candidate that the per-outlet quota skipped?It is deferred, not discarded: it stays in the candidate pool with its original relevance score and can take a later slot on the same page or the first slot on the next one. Nothing rescores it, and it is not removed from the candidate source - it simply lost this slot to the cap.
- Why does this layer force the funnel to retrieve more candidates than there are slots?Because every edit here is subtractive: de-duplication collapses clusters, eligibility filters remove items, quotas defer them. A shortlist the size of the page would produce a short page as soon as one rule fires. The layer over-fetches - a shortlist of hundreds for a slate of ten - so the rules have room to remove without underfilling.
- Who can see, in production, that a quota fired?Only whoever added per-rule logging. The served slate looks like an ordinary ranked list, so unless the layer records which rule deferred or removed which candidate at which slot, a later question - why did this story not appear - has no answer beyond re-running the request and hoping the same rules fire.
A page editor gets a stack of stories already ranked by how interesting they are, then lays out the page under house rules - no more than two from one wire service. The ranking of the stack is untouched; only the layout changes.
saying these in an interview costs you the question
- Says the ranking model itself learned to spread outlets across the page
- Thinks a skipped candidate has its relevance score lowered
- Believes the quota is applied at retrieval by fetching two per outlet
- Assumes the client hides the extra items after the page renders
- Cannot say which stage sees the slate as a set rather than one item