What are the maximum likelihood estimates of the mean and variance of a Normal sample?

## The log-likelihood The Normal density with mean `mu` and variance `sigma^2` is `f(x) = (1 / sqrt(2*pi*sigma^2)) * exp(-(x - mu)^2 / (2*sigma^2))` With `n` independent observations the likelihood is the product of these, and taking logs turns it into `l(mu, sigma^2) = -(n/2)*log(2*pi) - (n/2)*log(sigma^2) - (1/(2*sigma^2)) * sum((xi - mu)^2)` Two unknowns means two score equations, solved one after the other. ## Solving for the mean Only the last term contains `mu`: `d l / d mu = (1/sigma^2) * sum(xi - mu)` Set it to zero. The factor `1/sigma^2` is strictly positive, so it cancels, leaving `sum(xi - mu) = 0`, hence `sum(xi) = n*mu`, hence `mu-hat = xbar` Two things are worth saying out loud. First, the variance disappeared from this equation, so the maximum likelihood mean does not depend on knowing or estimating `sigma^2` -- the two problems partly decouple. Second, minimising `sum((xi - mu)^2)` over `mu` is the same algebra, which is the first hint that squared-error fitting and Normal likelihoods are the same object viewed from different sides. ## Solving for the variance Treat `v = sigma^2` as the parameter (differentiating in `v` rather than `sigma` keeps the algebra clean): `d l / d v = -n/(2*v) + (1/(2*v^2)) * sum((xi - mu)^2)` Setting to zero and multiplying through by `2*v^2` gives `-n*v + sum((xi - mu)^2) = 0`, so `v-hat = (1/n) * sum((xi - mu)^2)` Substituting the already-solved `mu-hat = xbar`: `sigma^2-hat = (1/n) * sum((xi - xbar)^2)` The estimate is the mean squared deviation from the sample mean. ## Why the divisor is n here Candidates who have memorised "the sample variance divides by n-1" often reflexively write `n-1` and are then unable to say where it came from. The honest answer is that the maximisation does not produce it. Setting the derivative of the Normal log-likelihood to zero returns `n`, full stop -- there is no step in the calculus where an `n-1` can appear. The `n-1` divisor is the answer to a different question, about how the estimator behaves across repeated samples, and it is decided by a different criterion than maximising the likelihood. If you are asked "which is right?", the correct framing is that they optimise different objectives, so both are right about their own objective. On this leaf the point to nail is simply: the likelihood's answer is `n`. ## A worked check Take the four values 2, 4, 4, 6. Then `xbar = 4`, and the squared deviations are 4, 0, 0, 4, summing to 8. The maximum likelihood variance is `8/4 = 2`, and hence the maximum likelihood standard deviation is `sqrt(2) ~ 1.41`. Notice that the second step read the standard deviation straight off the variance estimate rather than re-running the maximisation; that shortcut is the invariance property of maximum likelihood. ## Confirming it is a maximum As a function of `mu` with `v` fixed, the log-likelihood is a downward parabola, so `xbar` is clearly its maximum. As a function of `v` with `mu` fixed at `xbar`, the log-likelihood behaves like `-(n/2)*log(v) - S/(2*v)` where `S = sum((xi - xbar)^2)` -- it tends to minus infinity as `v` approaches 0 and as `v` grows large, so its single stationary point at `v = S/n` is the maximum. ## Degenerate cases If every observation is identical then `S = 0` and the maximiser sits at `sigma^2-hat = 0`, a boundary point where the Normal density is not defined; the likelihood is unbounded there. With `n = 1` the same thing happens, since a single point has zero deviation from its own mean. These are the standard reminders that a variance estimate needs more than one distinct observation to mean anything, and that maximum likelihood can run off the edge of the parameter space when the model is asked for more than the data contain. ## What interviewers listen for A correct pair of estimates, the observation that `sigma^2` cancels out of the mean equation, and a calm, accurate explanation of the `n` divisor rather than a panicked switch to `n-1`. Getting that last part right signals that you derived the result rather than recalled it.