What must a marketplace's signed-off policy on wrongly removing honest sellers versus leaving fraudulent listings up actually state?
answer
- someone signs, the system implements
- a ratio, not a sentiment
- a rate, not a running total
- name the instrument that measures it
- prefer the reversible action when uncertain
basics
~20 sAn accepted exchange rate between the two errors, owned by the business rather than the modelling team; a wrongful-removal ceiling stated as a measurable rate; whether the rate differs by seller tenure; the appeal path and its target; and who reviews it as the mix changes.
solid answer
~40 sThis is an input the platform signs, not an output the system derives. The policy has to name how much abuse loss the marketplace will accept rather than wrongly remove one established seller, and say who owns that number — risk and the business, not the people who train the model. It needs a ceiling expressed as a rate with a named measurement, such as upheld appeals per ten thousand sellers a month, plus what happens when the ceiling is breached. It should say whether the exchange rate differs by tenure and admit that as a deliberate choice, require the reversible action where the score is uncertain, and fix an appeal path with its own time target. Finally it needs a review cadence, because the number only stays true while the traffic mix does.
go deeper
Recall that the two errors do not cost the same and that the ratio between them is a business decision written down somewhere, not something the model works out.
Explain how the policy becomes mechanism: which score bands map to which action, and why the uncertain band should take the most reversible action available.
Show how it is measured and enforced — a rate with a named instrument, audits alongside appeals, and a defined breach procedure with an owner.
Own the number itself: who signs it, what it costs in accepted loss, how segmentation by tenure is justified, and what forces a review when the mix moves.
## Why this is a signed policy, not a derived number Every automatic action on a marketplace makes a standing trade: some honest sellers will be limited or removed, and some fraudulent listings will be left up. A system cannot discover the right ratio for itself, because the two costs are not in the same units — one is money lost to abuse, the other is a seller's livelihood, a support cost, a regulatory exposure and a reputational hit. Someone with the authority to accept that has to write it down, and the trust-and-safety system then implements it. When nobody does, the ratio still exists: it is whatever fell out of the last threshold change, owned by no one and never reviewed. ## What the policy must state 1. **The accepted exchange rate.** How much abuse loss the platform will absorb rather than wrongly remove one established seller — stated as a number the team may act on, with the seller segment it applies to. 2. **Who owns it.** Risk and the business sign; the modelling and platform teams implement and report. A policy owned by whoever last tuned a threshold is not a policy. 3. **A ceiling as a rate, not a total.** For example upheld appeals per ten thousand active sellers per month. A total grows with the platform and quietly loosens; a rate does not. 4. **The measurement.** Name the instrument: reinstatement rate after appeal, sampled audits of automatic removals, or both. An unmeasurable ceiling is a sentiment. 5. **Segmentation, stated openly.** A seller with three years of history and a week-old account are not treated alike, and if the exchange rate differs by tenure or value, the policy says so rather than letting the feature set imply it. 6. **The breach procedure.** What happens when the ceiling is exceeded: which tier reverts to a reversible action, who is notified, and by when. 7. **The appeal path and its target.** Who reviews an appeal, how long it may take, and what the seller's state is while it runs. 8. **A review cadence.** The accepted rate is only valid for the mix it was set on; a changed seller base or a new abuse pattern makes it stale. ## The reversibility ladder The policy's most useful operational clause is usually about **which action to take when uncertain**, not about the number: - **Fully reversible, self-service**: a verification challenge or a held payout. The seller clears it without anyone intervening. - **Reversible with a human**: a demotion or a listing suspension, undone by a reviewer or an appeal. - **Reversible on paper only**: an account removal. The account can be restored, but the buyers, ranking history and momentum do not come back. Stating the preference — in the uncertain band, take the most reversible action that contains the risk — turns the cost asymmetry into something an engineer can implement without re-deriving it per feature. ## Where the policy meets the queue The policy also has to say what happens when it collides with capacity, because it will. If a case would be reviewed under the policy but the shift is full, one of three things must be named as the answer: surge the shift, apply the reversible action and wait, or accept the loss. Leaving it unnamed means the queue's overflow behaviour decides, and that decision is made by whichever cases happened to expire. ## Measuring it honestly | Quantity | How it is actually read | Trap | |---|---|---| | Wrongful removals | Upheld appeals, plus audits of removals never appealed | Most wrongly removed sellers never appeal, so appeals alone understate it | | Abuse left up | Buyer complaints and settled disputes, weeks later | Arrives too late to gate today's decision | | Friction cost | Challenge completion rate by seller segment | A tier that is never appealed can still be declining honest sellers quietly | The asymmetry in the traps is the point: wrongful removals are undercounted by the instrument that measures them, so a ceiling read only from appeals should carry a deliberate margin. ## What a good answer sounds like A strong candidate resists two temptations. The first is to make this a modelling question — the policy is the input, and what is done with the fitted score afterwards is a separate craft. The second is to leave it qualitative: "we care more about false removals" is not a policy, because no one can implement it, breach it or review it. The test is whether an engineer could read the document and place the tiers, and whether an executive could read the same document and know what they have agreed to.
- Why state the wrongful-removal ceiling as a rate rather than a monthly count?Because a count loosens silently as the platform grows: the same number against twice the sellers is half the protection. A rate per ten thousand active sellers per month stays comparable across growth and across seasons, and it can be compared between segments, which is where the real differences hide.
- The policy says wrongful removals must stay under a ceiling, measured by upheld appeals. What is wrong?Appeals undercount. Many wrongly removed sellers never appeal — they do not know how, or they leave — so the measured rate is a lower bound on the true one. Fix it by sampling removals for audit independently of whether anyone complained, and by carrying a deliberate margin under the ceiling.
- How often should the accepted exchange rate be revisited?On a fixed cadence and on a trigger. The number was set against a particular seller mix, abuse pattern and product surface, so a material change in any of them invalidates it; so does a breach of the ceiling. A quarterly review with a named owner, plus an out-of-cycle review after a policy breach or a new abuse wave, is the usual shape.
saying these in an interview costs you the question
- Leaving the ratio implicit in whoever last moved a threshold
- Stating the ceiling as a total rather than a rate
- Treating upheld appeals as a complete count of wrongful removals
- Assuming the accepted rate stays valid as the seller mix changes
- Handing the trade-off to the modelling team to decide
- Writing a preference in words with no measurement attached