skip to content

A marketplace's fake-review flag volume triples during a seasonal spike: hold the score cutoff fixed, or hold daily review volume fixed?

level: seniorimportance: should knowfreq 48%

answer

  1. the shift does not scale with traffic
  2. fixed cutoff floats the intake
  3. bigger pool, same band, higher cutoff
  4. shed deliberately, ordered by risk
  5. a loss floor overrides the volume

basics

~20 s

Hold the volume. A fixed cutoff triples the queue's intake against an unchanged shift, so the backlog and time to decision blow out. Fixed volume raises the effective cutoff instead, trading a little more taken loss for decisions that stay on time.

solid answer

~40 s

The two policies fail differently. With a fixed score cutoff, a tripled pool sends roughly three times as many cases into a queue whose capacity did not change, so the backlog grows every day and cases expire unworked — the shift simply decides which ones by accident. Holding the volume fixed means taking the same top N by score, which raises the effective cutoff, so mid-band listings fall to an automatic tier instead of an analyst. That is the controlled version of the same shortfall. Do it with a floor: an expected-loss level that must be reviewed anyway and triggers surge, plus a widened challenge tier so automation, not people, absorbs the spike.

go deeper

for a junior

Recall that the analyst shift is fixed while flagged volume is not, so one of the two — the score cutoff or the daily case count — has to move in a spike.

for a middle

Work the numbers: a tripled pool at a fixed cutoff triples intake against unchanged capacity, and holding volume instead raises the effective cutoff.

for a senior

Argue for deliberate shedding — admission control at the low-risk end, a loss floor that overrides volume, and an emitted action recorded for everything shed.

for a principal

Decide in advance what the season buys: surge staffing, a wider automated challenge tier, or accepted loss, and get the number signed before the queue is days behind.

## Two policies, two failure modes A seasonal sale triples the number of listings and reviews that clear the cheap screen. The shift did not triple. Something has to give, and the design choice is **what**. | Policy | What stays constant | What gives | |---|---|---| | Fixed score cutoff | Which score enters review | Time to decision, then correctness, as the backlog swallows the queue | | Fixed review volume | The daily case count the shift works | The effective cutoff rises, so mid-band cases drop to an automatic tier | ## The arithmetic of a fixed cutoff Take the routine numbers: 9,000 scored candidates a day, a bound placed so that 480 enter review, a shift with 600 nominal capacity. Now the pool triples to 27,000 and the bound does not move. Roughly **1,440** cases a day enter a queue that closes 600, so the backlog grows about **840 a day**. Across a five-day sale that is around **4,200 open cases**, which at 600 a day takes another week to drain. Time to decision goes from hours to days, every aged case eventually hits its deadline, and the expiry action — whatever it is — becomes the platform's real policy for the sale. Worse, expiries are ordered by whatever the queue does last, not by risk. ## What fixed volume actually does Holding the volume means taking the top 480 by score each day out of 27,000 instead of out of 9,000. That is the top **1.8%** rather than the top 5.3%, so the **effective cutoff rises**. Get this direction right: a bigger pool with a fixed review capacity means a *higher* score is needed to see an analyst, and the listings that would have been reviewed last week now fall into the tier below. That is not free — it is the same shortfall, priced deliberately: - Cases dropping out of the band land in the challenge or auto-allow tier, so more abuse stays up during the sale. - The queue keeps its time-to-decision target, so honest sellers are not left in limbo at exactly the moment their sales matter most. - The shortfall is **visible and ordered by risk** rather than scattered by whatever expired. ## Why capacity usually wins A staffed queue is the one component in this architecture that does not scale with traffic. Letting its intake float is the equivalent of removing the admission control from an overloaded service: everything slows, nothing is shed on purpose, and the eventual shedding is random. Holding the volume is admission control — it sheds the lowest-risk end of the band first, which is exactly the end you would choose. ## The floor that stops it becoming a silent auto-allow Fixed volume alone will ration the wrong case on a bad day, so pair it with a rule that overrides volume: 1. Any case whose **expected loss** (score times exposure) exceeds an absolute floor enters review regardless of the day's count. 2. Breaching the floor's own budget triggers **surge**, not silence: overtime, borrowed reviewers from an adjacent team, or an explicit written acceptance that the excess rides. 3. Everything shed below the band gets the tier below it as an emitted action, recorded, so the sale's extra taken loss can be counted afterwards. ## Levers other than the cutoff The cutoff is one dial of several, and it is rarely the best one: - **Widen the challenge tier.** Verification and held payouts scale with traffic and cost no analyst time; most of a seasonal spike can land there instead of on people. - **Raise throughput, not headcount.** Pre-fetched evidence panels, clustering duplicates of the same seller ring into one case, and bulk actions on a cluster all raise cases per hour. - **Pre-book the surge.** A seasonal spike is predictable to the week; staffing agreed in advance is much cheaper than a mid-sale scramble. - **Screen earlier.** A cheaper first-stage filter that removes obviously benign candidates keeps the scored pool from tripling in the first place. ## What to agree before the season Write down, before the sale: the standing review volume, the loss floor that overrides it, the action for everything shed below the band, the surge trigger and who authorises it, and the time-to-decision target being protected. The point of deciding in advance is that all four of these are decisions about accepted loss, and a queue that is already three days behind is the worst possible place to make them.

  • Does holding the review volume fixed lower or raise the effective cutoff during a spike?
    It raises it. The same number of cases now comes out of a much larger pool, so a candidate needs a higher score to make the top N. Listings that reached an analyst before the spike now fall to the tier below, which is why fixed volume must be paired with an expected-loss floor and a widened challenge tier.
  • What is the earliest signal that the fixed-cutoff policy is failing?
    Open cases and the age of the oldest open case, both rising day over day, before p95 time to decision breaches and long before anyone complains. Daily intake against daily closures is the same signal expressed as a rate: once intake exceeds closures for two consecutive days, the backlog is structural and the band has to be narrowed or the shift surged.
  • Why is widening the challenge tier often better than raising the cutoff?
    Because it absorbs the spike with automation instead of taking the loss outright. A shed case moved to auto-allow contributes its full expected loss; the same case moved to verification or a held payout still gets friction that deters a fraud ring and is self-service for an honest seller. The cost is friction on good sellers, so measure challenge completion rates.

saying these in an interview costs you the question

  • Assuming a fixed score cutoff keeps the review workload stable
  • Thinking a fixed review volume lowers the effective cutoff in a spike
  • Letting the backlog shed cases instead of shedding them on purpose
  • Treating a seasonal spike as unpredictable when it is on the calendar
  • Sizing to the spike year-round and leaving analysts idle otherwise
  • Shedding into auto-allow without recording the accepted loss