A $180,000 B2B order sits in a consumer marketplace's GMV column; do you drop, cap or keep it?

## The decision is about population, not about distance The common instinct is to reach for a numeric rule: anything beyond some percentile goes. That instinct answers the wrong question. The real question is **whether rows like this one belong to the population your model will be asked to score**. A value can be a hundred times the median and be perfectly legitimate; a value can be well within the bulk and still be corrupt. So work through four questions in order. ### 1. Is the value real? Check the source of truth. A GMV of $180,000 in a marketplace where the median order is $40 has three plausible explanations: a genuine bulk purchase, a unit or currency error, or a test transaction that leaked into production data. Only the first is an outlier. The second is a bug to fix upstream, and the third is a row to exclude by its own flag. This step costs one query and settles the question more often than people expect. ### 2. Will rows like it be scored in production? This is the pivotal question. If the model will see B2B orders at serving time, deleting them from training produces a model that has never observed the range in which your revenue concentrates. It will predict badly on precisely the rows worth the most money, and the offline metric will not show it, because you deleted those rows from the evaluation data too. That is the failure mode that turns *cleaning* into *silently scoping the model down*. If the extremes are in scope, the useful moves are about **representation** rather than removal: - Add the feature that explains the difference. Buyer type, contract flag, channel. Once the model can tell a B2B order from a consumer one, the value is no longer inexplicable. - Consider a separate model for the segment if it behaves differently enough and there is sample to support one. - Accept that a handful of rows cannot teach a model much on their own, and be explicit about that limitation rather than hiding it by deleting them. ### 3. If they are out of scope, exclude them by rule If the product genuinely only serves consumers and B2B traffic will never be scored, then excluding those rows is correct. But exclude them **by the business rule that defines the boundary** (buyer type is business), not by a numeric threshold. A percentile filter is a proxy that misfires in both directions: it deletes the genuine large consumer orders and keeps the small B2B ones. Write the rule down, count the rows it removes, and monitor that count over time. ### 4. If they stay but destabilise fitting, cap the feature, not the target Capping is the middle path: the row is kept, its value is flattened to a bound. It is a reasonable choice when the value is genuine but you only need the model to behave sensibly across the bulk range and its accuracy on the far tail is not what you are optimising. One asymmetry decides most of these cases: **capping a feature limits the influence of an input; capping the target changes what the model is allowed to say**. If you winsorise GMV as a target at the 99th percentile, the model becomes structurally unable to predict anything above that bound, and every large-order prediction is biased low. It is not merely inaccurate, it is incapable. That is only acceptable when the business explicitly does not care about the tail, which is rarer than people assume. ## The evaluation set is not yours to clean Whatever you do to the training data, the validation and test sets must keep the extremes they naturally contain. If you remove them from evaluation as well, the reported error describes a population you will never face, and it will look better than production every time. Preprocessing does apply to evaluation rows, but only using bounds fitted on training data; **row removal is a different act from transformation** and does not carry over. A good habit is to report error separately for the bulk and the tail. That makes the cost of a cap explicit: you can see exactly how much accuracy on large orders you traded for stability on the ordinary ones, and let the business decide whether that trade is acceptable. ## Documenting the decision Whatever you choose, record it as part of the model definition: the rule, the number of rows affected, and the reason. Two failure modes follow from skipping this. Somebody re-runs the pipeline six months later without the filter and cannot reproduce your numbers. Or the filter silently starts removing 8% of rows instead of 0.2% because the business changed, and nobody notices because it was a percentile with no alarm attached to it. ## The short version Drop when corrupt or provably out of scope, by rule. Cap when genuine but disruptive, and cap inputs before you ever consider capping the target. Keep, and represent, when the row is genuine and in scope, which is more often than the reflex suggests. And never touch the evaluation set.