What does rank-normalising a heavy-tailed page-load-time feature gain and cost?

## The mechanics Rank normalisation (also called a quantile or rank-based transform) works in three steps: 1. **Rank.** Sort the n training values and assign each row its rank `r` from 1 to n. Tied values get the average of the ranks they span, otherwise the transform manufactures an ordering the data does not contain. 2. **Map to a uniform scale.** A standard plotting position is `(r - 0.5) / n`, which spreads the values evenly over (0, 1). The transformed column is now uniform by construction, whatever the original shape was. 3. **Optionally map to a normal shape.** Push each uniform value through the inverse normal CDF. The result is the rank-based inverse normal transform: an approximately standard-normal feature with no tail at all. This is the version people usually mean when they say they 'quantile-normalised a feature to a Gaussian'. ## What it buys **Distribution-free tail control.** Box-Cox and Yeo-Johnson assume the tail belongs to a power family and estimate one parameter. A latency distribution shaped by timeouts, retries and a hard ceiling often does not. The rank transform makes no assumption at all: whatever comes in, a uniform or Gaussian comes out. For a heavy-tailed feature entering a penalised linear model, a k-nearest-neighbour distance, or a k-means objective, that is a strong practical guarantee. **Immunity to individual extremes.** One absurd value moves by one rank position instead of stretching the whole scale. ## What it costs **Spacing is destroyed.** This is the central tradeoff. The transform preserves order and nothing else: two adjacent values in the sorted list end up the same distance apart whether they differed by 5 ms or by 30 seconds. If the magnitude of the gap carries signal — and for latency it very often does, since a 40-second load is a different failure mode, not a slower success — the transform deletes exactly the thing you cared about. **Interpretability is gone.** A coefficient on a rank-normalised feature reads 'per unit of the normal-scale rank', which no stakeholder can act on. Recovering an original-scale statement requires mapping back through the quantile function. **The mapping is fitted, and it must be applied consistently.** The transform is defined by the training sample's empirical quantile function. At scoring time you interpolate a new value between the stored training quantiles. Two consequences follow. First, fitting the quantiles on training and test data together is leakage, because the test distribution then shapes the feature definition. Second, a value above the training maximum has no rank; the usual handling is to clamp it to the top of the range, which means an unprecedented 5-minute stall looks identical to the worst training case. If detecting novel extremes matters, keep a raw or capped copy of the feature alongside. **Batch dependence.** If you compute ranks per batch rather than through a stored mapping, the same raw value produces different features in different batches — a classic serving-skew bug, because the feature depends on which other rows happened to be scored with it. ## Where it makes no difference The transform is strictly increasing, so it permutes the numeric thresholds an axis-aligned decision tree could pick but not the ordering of the rows. Every partition reachable before is reachable after, so the fitted tree is unchanged. Applying a rank transform to features for a random forest or a boosted ensemble is wasted work; it matters only for models that read magnitude — linear and penalised linear models, distance-based methods, and anything with a squared-error term on that input. ## Choosing between rank and a power transform Use a **power transform** (Box-Cox for strictly positive, Yeo-Johnson for signed data) when the spacing carries information, when you want an invertible mapping with a single interpretable parameter, and when the tail is heavy but not pathological. Use a **rank transform** when the tail defeats a parametric family, when you only trust the ordering of the measurements anyway, or when a downstream method needs bounded, well-conditioned inputs more than it needs faithful magnitudes. A common middle path is to keep both: the rank version for the model that needs stability, the raw version for reporting and for alerting on true extremes.