Random Variables & Moments

Multiply each outcome by its probability and add the pieces up. On a 38-pocket wheel a $1 straight-up bet nets +$35 with probability 1/38 and -$1 with probability 37/38, giving about -$0.053 per dollar staked.

Exactly zero. Under a continuous model, any single point has zero width and therefore zero area under the density, so only intervals carry probability. Ask instead for P(199.5 < X <= 200.5).

KL(P||Q) = sum P(x) log(P(x)/Q(x)) is the cost of describing draws from P as if they came from Q. It is never negative and zero only when P equals Q, but it is asymmetric, so it is a divergence, not a metric.

Linearity of expectation is proved by regrouping a sum over the joint distribution, never by multiplying probabilities. E[X + Y] = E[X] + E[Y] holds for any variables with finite means. Independence is needed for products, not sums.

Covariance carries the units of X times the units of Y, so rescaling either variable rescales it. Its sign shows the direction of linear association; only the correlation Cov(X,Y)/(sd(X)*sd(Y)), bounded in [-1,1], measures strength.

Covariance detects only linear association. A pair can be perfectly dependent through a curved relationship and still have zero covariance: with X uniform on (-1,1) and Y = X^2, Cov(X,Y) = 0 even though Y is a function of X.

A density is probability per unit of x, not a probability. Probability is the area under the curve, so a tall density over a narrow interval still has total area 1. The uniform density on [0, 0.5] equals 2 everywhere.

By convolution: enumerate every split of the target value and multiply the two probabilities, which independence permits. Discretely, P(X+Y=s) = sum over k of P(X=k)*P(Y=s-k); continuously, f_S(s) = integral of f_X(x)*f_Y(s-x) dx.

Moment generating functions multiply for independent variables, and the product of two normal moment generating functions is again a normal one. The sum is normal with the two means added and the two variances added.

Sum a row or column of joint probabilities to get a marginal: P(mobile) = P(mobile, free) + P(mobile, pro). Divide a single joint cell by that marginal to get a conditional: P(pro | mobile) = P(mobile, pro) / P(mobile).

Go through the CDF. The maximum is at most x exactly when every draw is at most x, so F_max(x) = F(x)^n for n independent draws with CDF F. The minimum instead gives F_min(x) = 1 - (1 - F(x))^n.

Cross-entropy splits into entropy plus divergence: H(P,Q) = H(P) + KL(P||Q). Entropy is the unavoidable cost of describing draws from P, KL is the extra cost of using Q instead, and cross-entropy is the total.

It measures how many bits knowing the channel removes from the uncertainty about conversion: I(X;Y) = H(Y) - H(Y|X). It is symmetric, never negative, and exactly zero when channel and conversion are independent.

Variance is multiplied by 1.8 squared, which is 3.24, and the plus-32 shift changes nothing. In general Var(aX + b) = a^2 * Var(X). The standard deviation is multiplied by 1.8, so 2 degrees Celsius becomes 3.6 degrees Fahrenheit.

Expand the squared deviation inside the expectation. Var(X) = E[(X - mu)^2] becomes E[X^2] - 2muE[X] + mu^2, and since E[X] = mu the last two terms collapse to -mu^2, leaving E[X^2] - (E[X])^2.

Invert the CDF rather than reading heights: the median is the smallest t with F(t) at least 0.5, and the p90 the smallest t with F(t) at least 0.9.

c = 1/2. The area under c*x from 0 to 2 is c times 2, and a density's total area must equal 1, so c = 1/2. The sign is fixed by requiring the density to be non-negative.

A density must be rescaled by the derivative of the inverse map. The inverse of Y = e^X is X = ln y, whose derivative is 1/y, so f_Y(y) = f_X(ln y)/y for y > 0.

KL is infinite whenever the current month has mass in a category the reference month gives zero probability. A brand-new channel or an empty bin makes the log ratio diverge, so the score explodes for a reason unrelated to drift size.

Because the reciprocal function is strictly convex on positive values, Jensen's inequality gives E[1/X] >= 1/E[X], strict unless X is constant. Averaging per-item rates therefore overstates the rate implied by the average latency.

With both streams at variance s^2 and correlation rho, the blend has variance 0.5s^2(1 + rho). Perfectly correlated streams give no reduction, uncorrelated ones halve the variance, and rho = -1 cancels it entirely.

About 225 packs, if each pack holds one sticker drawn uniformly at random. Split the hunt into 50 stages with expected lengths 50/50, 50/49, ..., 50/1, which sum to 50 times the 50th harmonic number.

Y given X = x is again normal, with mean mu_Y + rho*(sd_Y/sd_X)(x - mu_X) and variance sd_Y^2(1 - rho^2). The mean moves linearly with x, and the spread does not depend on x at all.

As a mixed distribution: no single PMF or PDF describes it. Use the CDF, which jumps by the non-buyer share at 0 then rises smoothly, or a mixture of an atom and a spend distribution.

It is a sum of squares: if Z1 through Zk are independent standard normals, then Z1^2 + ... + Zk^2 is chi-square with k degrees of freedom, never negative, with mean k and variance 2k.

Pick the direction by which error you can least afford. Minimising KL(P||Q) is mass-covering: the approximation stretches to cover everything the target produces. Minimising KL(Q||P) is mode-seeking: it locks onto one region and ignores the rest.