skip to content

A statue scaled non-uniformly has lighting normals tilted the wrong way; why does transforming normals by the model matrix fail, and what fixes it?

level: seniorimportance: should knowfreq 56%

answer

  1. normals stay perpendicular to tangents
  2. scale stretches tangents and normals differently
  3. inverse, then transpose
  4. renormalise afterwards

basics

~10 s

Non-uniform scale stretches the surface so a normal pushed through the model matrix is no longer perpendicular to it. Transform normals with the inverse-transpose of the model matrix's upper-left 3x3 block, then renormalise.

solid answer

~40 s

A normal is defined by being perpendicular to every tangent on the surface, `dot(n, t) = 0`. Tangents lie along the surface, so they transform like the mesh: `t' = M * t`. Requiring `dot(n', M * t) = 0` for every `t` gives `n' = transpose(inverse(M)) * n`, the **normal matrix**, using only the 3x3 part because normals are directions. For a pure rotation the inverse-transpose equals the matrix, and for a uniform scale it differs only in length, which is why the bug appears only once a statue is stretched along one axis. The fix is to compute the inverse-transpose once per object, transform normals with it and renormalise. A mirrored scale (negative determinant) also reverses triangle winding, which is a separate fix for culling order and tangent-frame handedness.

code

pseudocode · 8 lines
pseudocode
// once per object; column vectors
M3 = upperLeft3x3(modelMatrix)
N  = transpose(inverse(M3))   // the normal matrix

// per vertex
positionWorld = modelMatrix * (position, 1)
tangentWorld  = normalize(M3 * tangent)
normalWorld   = normalize(N * normal)

go deeper

for a junior

Recall that normals must stay perpendicular to the surface and that non-uniform scale breaks this if they are transformed like positions.

for a middle

Explain that normals use the inverse-transpose of the 3x3 block, why rotation and uniform scale do not need it, and why you renormalise.

for a senior

Derive it from dot(n, t) = 0, diagnose the symptom with debug lines and a uniform-scale test, and handle mirrored scale and tangent frames.

for a principal

Decide whether non-uniform scale is allowed on content at all, and where the normal matrix is computed and cached so every system shares one correct path.

## The symptom A statue stretched to twice its width along one axis renders with lighting that looks wrong on its sloped faces: highlights sit in the wrong place, and slopes that should face the light look dim. With a uniform scale or only a rotation, the same statue lights correctly. The vertex positions are right, so the model matrix itself is fine; what is wrong is how the **normals** — unit vectors perpendicular to the surface, used for lighting — were transformed. ## Why normals are different The convention here is **column vectors** (`p' = M * p`), and `M` is the upper-left 3x3 block of the model matrix; translation never matters because normals are directions with `w = 0`. 1. A **tangent** is a direction lying in the surface, such as an edge of a triangle. Tangents are differences of points on the mesh, so they transform exactly like the mesh: `t' = M * t`. 2. A normal is defined by perpendicularity: `dot(n, t) = 0`, written as a matrix product `n^T * t = 0`. 3. After the transform the new normal must satisfy `n'^T * (M * t) = 0` for every tangent `t`. 4. Insert the identity: `n^T * t = n^T * inverse(M) * M * t`. So choosing `n'^T = n^T * inverse(M)` keeps the product zero. 5. Transposing both sides: `n' = transpose(inverse(M)) * n`. That matrix, `N = transpose(inverse(M))`, is the **normal matrix**. Transforming normals with `M` itself only works when `M` happens to equal its own inverse-transpose up to a scale factor. ## Worked numbers Let the statue's scale be `S = diag(2, 1, 1)`: twice as wide along `x`. Take a 45-degree slope with normal `n = (1, 1, 0)` (before normalising) and a tangent along the slope `t = (1, -1, 0)`; `dot(n, t) = 0`. 1. The tangent becomes `S * t = (2, -1, 0)`. 2. The naive normal is `S * n = (2, 1, 0)`. Its dot product with the new tangent is `4 - 1 = 3`, not zero: not perpendicular. 3. The inverse-transpose is `diag(0.5, 1, 1)`, giving `(0.5, 1, 0)`. Its dot product with the tangent is `1 - 1 = 0`: perpendicular. In angles, the stretched slope's tangent points about 26.6 degrees below the `x` axis, so the true normal sits at about 63.4 degrees above it. The naive normal sits at about 26.6 degrees: tilted 36.9 degrees too far toward the stretched axis, which is exactly the visible error. ## When the plain model matrix is enough | 3x3 part of the model matrix | inverse-transpose | can normals use M directly? | |---|---|---| | rotation `R` | `R` | yes | | uniform scale times rotation, `s * R` | `R / s` | yes, after renormalising | | non-uniform scale, `R * S` | `R * S^-1` | no | | mirrored scale (a negative factor) | still the correct direction | the normal is right, but triangle winding flips | ## Production notes - **Compute it once per object**, on the CPU or once per draw, not once per vertex: a 3x3 inverse is cheap but not free, and every vertex shares it. - **Renormalise** after transforming. Even the correct normal matrix changes lengths whenever there is any scale. - **Tangents use `M`**, normals use the normal matrix; a bitangent is usually rebuilt as a cross product of the two, with a sign stored per vertex. - **The cofactor shortcut.** The **cofactor matrix** of `M` equals `det(M) * transpose(inverse(M))`, so it points the same way without a division and stays defined when a scale factor is zero. When `det(M)` is negative it points the opposite way, so take its sign into account. - **Mirroring.** A negative scale factor reverses the winding of every triangle, so the renderer's front-face test and the tangent frame's handedness sign must flip; the normal matrix alone does not fix those. ## Diagnosing it - Draw the transformed normals as short debug lines. On a stretched statue the wrong ones lean toward the stretched axis instead of standing perpendicular to the faces. - Set the scale to uniform. If the lighting becomes correct, the normal transform is the culprit, not the light or the material.

  • Why can you skip the inverse-transpose for rotations and uniform scale?
    For a rotation R the inverse is R^T, so the inverse-transpose is R itself. For s * R it is R / s, the same directions at a different length, and renormalising removes the length. Only non-uniform scale or shear changes the direction.
  • How do tangent vectors transform, compared with normals?
    Tangents lie in the surface, so they transform with the model matrix itself, like edges of the mesh. Normals use the inverse-transpose. The bitangent is usually rebuilt from the cross product of the transformed normal and tangent, times a stored handedness sign that flips when the scale mirrors the mesh.
  • What is the cofactor-matrix alternative to the inverse-transpose, and what is its catch?
    The cofactor matrix equals det(M) times the inverse-transpose, so it gives the same normal direction without dividing by the determinant and still works when a scale factor is zero. The catch is the sign: with a negative determinant it points the opposite way, so it must be corrected for mirrored transforms.

  • Normals are directions, so w = 0 through the model matrix is enough
  • Renormalising after the model matrix makes the normal correct
  • The inverse-transpose is needed because of translation
  • The normal matrix is just the inverse of the model matrix
  • Non-uniform scale changes a normal's length but never its direction