Why does the mean of 1/X exceed 1 divided by the mean of X for a random latency X?

## The inequality A function g is convex when its second derivative is non-negative wherever it is defined, meaning the curve bends upward. Jensen's inequality states that for such a g and any random variable X with finite mean, `E[g(X)] >= g(E[X])` If g is concave, the curve bends downward and the inequality flips to `E[g(X)] <= g(E[X])`. If g is affine, that is `g(x) = ax + b`, both hold and the relation is an equality: this is exactly the linearity case, and it is the only case in which you may freely move an expectation inside a function. For `g(x) = 1/x` on positive x, the second derivative is `2/x^3`, which is positive, so the reciprocal is strictly convex and `E[1/X] >= 1/E[X]` with equality only if X takes a single value with probability 1. ## A concrete gap Suppose a request takes 1 second with probability 0.5 and 3 seconds with probability 0.5. - Mean latency: `E[X] = 2` seconds. Reciprocal of the mean: `1/E[X] = 0.5` requests per second. - Mean of the rates: `E[1/X] = 0.5*(1/1) + 0.5*(1/3) = 0.6667` requests per second. The two differ by a third. Widen the spread and the gap explodes: with latencies of 0.1 and 3.9 seconds the mean is still 2 seconds, but `E[1/X] = 0.5*10 + 0.5*0.2564 = 5.13` per second, ten times the reciprocal of the mean. The asymmetry comes from the shape of `1/x`: near zero the reciprocal blows up, while for large x it flattens toward zero, so a few very fast observations dominate the average of rates while contributing almost nothing to the average of durations. ## Why this matters when reporting The two quantities answer different questions, and quoting one when the audience wants the other is a real and common reporting error. - `1/E[X]` is the throughput of a single sequential worker: how many items you get through per unit time, since total time is what limits you. - `E[1/X]` is the unweighted average of per-item rates, which gives a fast item the same weight as a slow one even though the fast one occupied a sliver of the window. The same trap appears whenever a ratio is averaged across units: per-user conversion rates averaged across users, per-region error rates averaged across regions, per-query cost averaged across queries. In each case the average of ratios is not the ratio of the totals, and the difference is not a rounding artefact but a systematic bias whose direction is fixed by the curvature of the map. The fix is to decide which question you are answering. If you want the aggregate rate over a window, aggregate the numerator and the denominator separately and divide once: total items divided by total time. If you genuinely want each unit weighted equally, an average of ratios is the right object, but then say so, because the number will not reconcile with the aggregate and someone will eventually ask why. ## The general pattern Once you see the shape of the argument you can predict the direction of the bias for any non-affine transformation without doing algebra: - Squaring is convex, so `E[X^2] >= (E[X])^2`. That is the same fact as variance being non-negative. - The logarithm is concave, so `E[log X] <= log E[X]`. Averaging on a log scale and then exponentiating gives something no larger than the plain average. - The exponential is convex, so `E[e^X] >= e^(E[X])`. - The maximum of a variable and a constant is convex, so `E[max(X, c)] >= max(E[X], c)`. This is why plugging a mean into a payoff with a floor understates the payoff. The last one is the general lesson for planning: substituting an average input into a non-linear model does not give you the average output. Feeding mean traffic into a queueing formula understates mean delay, because delay curves upward in load. ## How big is the gap? The size of the Jensen gap grows with both the curvature of g and the spread of X. A second-order approximation around the mean, when X is concentrated enough for it to be meaningful, gives `E[g(X)] approximately g(E[X]) + 0.5 * g''(E[X]) * Var(X)`. For the reciprocal this reads `E[1/X] approximately 1/mu + Var(X)/mu^3`, so the excess scales with the variance and blows up as the mean approaches zero. Two consequences: for a tight, well-behaved distribution the two numbers may agree to within noise, and for a heavy-tailed or near-zero-mass distribution they can differ by orders of magnitude, or `E[1/X]` may not even be finite. Knowing when the gap is negligible is as valuable in an interview as knowing that it exists.