What is the Hessian matrix of a scalar function, and when is it symmetric?

## What the Hessian is Take a scalar-valued function of several variables, `f(x1, x2, ..., xn)` — a loss as a function of parameters, say. Its **gradient** is the vector of first partial derivatives, `g = (df/dx1, ..., df/dxn)`, and it describes the local slope. Differentiating each of those first derivatives again with respect to each variable produces `n * n` second partial derivatives, and arranging them in a grid gives the **Hessian matrix**: ``` H[i][j] = d2f / (dxi dxj) ``` So `H` is `n x n`. The **diagonal** entries `H[i][i] = d2f/dxi^2` are pure second derivatives: how fast the slope along axis `i` is itself changing — the curvature of the 1-D slice you get by moving along that axis alone. The **off-diagonal** entries `H[i][j]` with `i != j` are **mixed partials**: how the slope in direction `i` changes as you move in direction `j`. A non-zero mixed partial means the two variables interact — the surface is tilted relative to the coordinate axes rather than being a clean sum of one-variable pieces. ## A worked example Let `f(x, y) = x^2 * y^3`. First derivatives: - `f_x = 2*x*y^3` - `f_y = 3*x^2*y^2` Second derivatives: - `f_xx = 2*y^3` - `f_yy = 6*x^2*y` - `f_xy = d/dy (2*x*y^3) = 6*x*y^2` - `f_yx = d/dx (3*x^2*y^2) = 6*x*y^2` So ``` H(x, y) = [[ 2*y^3 , 6*x*y^2 ], [ 6*x*y^2 , 6*x^2*y ]] ``` Two things are worth noticing. First, `f_xy` and `f_yx` came out identical even though the differentiations were done in opposite orders. Second, the Hessian is a **function of the point**, not a single fixed matrix: at `(1, 1)` it is `[[2, 6], [6, 6]]`, at `(1, 2)` it is `[[16, 24], [24, 12]]`. Curvature is local, exactly as slope is. ## Why symmetry holds The result that `d2f/dx dy = d2f/dy dx` is **Clairaut's theorem** (also called Schwarz's theorem). Its hypothesis matters: the mixed second partials must exist and be **continuous** in a neighbourhood of the point. Under that condition the order of differentiation is irrelevant and `H` is a symmetric matrix, `H = H^T`. It is not a triviality that can be assumed for free. There is a classic counterexample, `f(x, y) = x*y*(x^2 - y^2) / (x^2 + y^2)` with `f(0, 0) = 0`, whose two mixed partials at the origin exist but differ in sign; the reason is precisely that they fail to be continuous there. In practice, the losses and models you differentiate for optimisation are smooth enough on the region of interest that symmetry holds, and every treatment of curvature quietly relies on it. The honest interview answer is "symmetric when the second partials are continuous, which is the normal case", not "always symmetric". A useful intuition for *why* it should hold: the mixed partial measures the second difference of `f` across a small rectangle with corners `(x, y)`, `(x+h, y)`, `(x, y+k)`, `(x+h, y+k)` — namely `f(x+h, y+k) - f(x+h, y) - f(x, y+k) + f(x, y)`, divided by `h*k`. That expression is manifestly unchanged if you swap the roles of the two variables, since it just re-groups the same four corner values. Taking limits in either order lands on the same number when things are well behaved. ## Why practitioners care The Hessian is the object that turns a first-order picture of a surface into a second-order one. The gradient tells you which way is downhill and how steeply; the Hessian tells you how that steepness is changing, i.e. whether the surface is a gentle trough, a tight ravine, or nearly flat. It is the coefficient of the quadratic term in a Taylor expansion, so it is the smallest amount of extra information beyond the gradient that lets you approximate a surface by a bowl rather than a plane. Symmetry has practical consequences too. An `n x n` Hessian has `n^2` entries but only `n*(n+1)/2` distinct ones — for `n = 3` that is 6 numbers, not 9 — which halves both the storage and the number of second derivatives you actually have to compute or estimate. Symmetry also means the matrix behaves well under the standard machinery for symmetric matrices, which is what the linear-algebra treatment of curvature builds on. ## Dimensions and common slips Keep the shapes straight: for `f: R^n -> R`, the gradient is `n x 1` and the Hessian is `n x n`. The Hessian is *not* defined for a vector-valued function in this simple form — stacking second derivatives of a map `R^n -> R^m` gives a three-index object, not a matrix. And the Hessian is not the square of the gradient, nor the derivative of the function twice along one direction only; every pair of variables contributes an entry.