In the chain rule for f(g(h(x))), why is the total Jacobian the product J_f J_g J_h in that order?

## The statement Let `h: R^n -> R^p`, `g: R^p -> R^q` and `f: R^q -> R^m` all be differentiable, and let `F = f o g o h`, so `F: R^n -> R^m`. The chain rule says `J_F(x) = J_f(g(h(x))) * J_g(h(x)) * J_h(x)` with shapes (m by q)(q by p)(p by n) = m by n. ## Why the order is forced, argument one: shapes A matrix product `A B` is defined only when the number of columns of A equals the number of rows of B. Here `J_h` is p by n, `J_g` is q by p, `J_f` is m by q. Try the reversed order: `J_h J_g` would need n columns of the left factor to meet q rows of the right, which happens only by coincidence, and even when the numbers accidentally line up the resulting matrix does not have the meaning you want. The dimensions themselves tell you which side each Jacobian goes on: the innermost stage, the one that touches the original input, must sit at the far right, because it is the factor that will multiply the input displacement first. ## Why the order is forced, argument two: composition of linear maps The derivative is not fundamentally a matrix, it is a linear map: the best linear approximation to the function near a point. If `A` is the linear map approximating f and `B` the one approximating g, then the approximation to `f o g` is the composed map `A o B`, meaning apply B first and then A. In matrix language, composition of linear maps is matrix multiplication written in that same right-to-left order. So the chain rule is not a separate fact about derivatives; it is the statement that linearization respects composition. You can see it operationally too. Nudge the input by a small d. The first stage produces a change of about `J_h d`. The second stage sees that change as its own input and produces about `J_g (J_h d)`. The third produces about `J_f (J_g J_h d)`. Peeling the parentheses off gives the product in exactly the stated order. ## Non-commutativity is not a technicality Even when all the stages map R^k to R^k and every Jacobian is square, so that every ordering is conformable, the answer still depends on the order, because matrix products generally differ when swapped. There is no version of the rule where you may reorder the factors for convenience. This is the precise sense in which the single-variable mnemonic misleads: in one dimension every Jacobian is a 1 by 1 matrix, ordinary numbers commute, and so the du/dx-style cancellation appears to work. It is a coincidence of dimension one. ## Evaluation points Each Jacobian is a function of where you evaluate it, and the three factors are evaluated at three different places: - `J_h` at the original input x, - `J_g` at the intermediate value `h(x)`, - `J_f` at the second intermediate value `g(h(x))`. A derivation that writes `J_f J_g J_h` without saying this is only half correct, and a numerical implementation that evaluates all three at x will produce plausible-looking garbage. In practice this means a forward pass through the stages must record the intermediate values, because the derivative factors need them. ## A worked shape check Suppose `h: R^5 -> R^3`, `g: R^3 -> R^2`, `f: R^2 -> R^4`. Then `J_h` is 3 by 5, `J_g` is 2 by 3, `J_f` is 4 by 2, and `J_F = J_f J_g J_h` has shape (4 by 2)(2 by 3)(3 by 5) = 4 by 5, matching a map from R^5 to R^4. Every adjacent pair agrees, and the outer dimensions reproduce the overall input and output sizes. That two-line check is the fastest way to validate any composed derivative before touching numbers. ## Associativity, and why it is useful Matrix multiplication is associative even though it is not commutative, so `(J_f J_g) J_h` and `J_f (J_g J_h)` give the same result. The grouping is free; the order is not. The grouping does change the arithmetic cost — for a scalar final output, multiplying from the left keeps every intermediate a single row, whereas multiplying from the right builds full matrices — which is why the associativity remark is worth knowing even when only correctness is being asked about. ## Answering well Give the shape argument first because it is short and checkable, then the linear-map argument because it explains rather than verifies, then name the evaluation points. Adding that associativity holds while commutativity does not is the detail that shows real command of the rule.