How do the singular values of a matrix A relate to the eigenvalues of A^T A?

## The derivation Start from the factorization `A = U S V^T` with U and V orthogonal and S diagonal with non-negative entries. Form the product `A^T A = (U S V^T)^T (U S V^T) = V S^T U^T U S V^T = V (S^T S) V^T` since `U^T U = I`. The matrix `S^T S` is n-by-n and diagonal, with entries `sigma_1^2, sigma_2^2, ...` (padded with zeros if n exceeds the number of singular values). So `V (S^T S) V^T` is a diagonalization of A^T A by an orthogonal matrix: its **eigenvalues are the squared singular values** and its **eigenvectors are the columns of V**. Inverting the relation: `sigma_i = sqrt(lambda_i(A^T A))` The same computation on the other side gives `A A^T = U (S S^T) U^T`, an m-by-m matrix whose eigenvalues are again the squared singular values and whose eigenvectors are the columns of U. ## Why the square roots are always real For any vector x, `x^T (A^T A) x = (A x)^T (A x) = ||A x||^2 >= 0`. So A^T A is symmetric positive semi-definite: every eigenvalue is real and at least zero, and every square root is a real non-negative number. This is the structural reason singular values can never be negative or complex, no matter what A looks like. ## A concrete tall example Take a 3-by-2 matrix A of rank 2. Then: - `A^T A` is 2-by-2 with two positive eigenvalues; their square roots are `sigma_1` and `sigma_2`, and its two orthonormal eigenvectors are `v_1` and `v_2`. - `A A^T` is 3-by-3. Its eigenvalues are `sigma_1^2`, `sigma_2^2` and `0`. The eigenvectors for the nonzero eigenvalues are `u_1` and `u_2`; the third spans the orthogonal complement of the column space. In general the two products always share their nonzero eigenvalues with identical multiplicities; they differ only in the number of extra zeros, and the larger of the two carries `|m - n|` more of them. That is why, for a tall matrix, the small product is the convenient one to look at. Having computed V and the nonzero `sigma_i`, the left singular vectors come for free: from `A v_i = sigma_i u_i` we get `u_i = A v_i / sigma_i` whenever `sigma_i > 0`. ## The symmetric special case If A itself is symmetric with eigenvalue `lambda_i`, then the singular values are `|lambda_i|`. A negative eigenvalue contributes a negative sign that is absorbed by flipping the corresponding left singular vector. The two spectra coincide exactly when A is symmetric positive semi-definite, and differ whenever a negative eigenvalue is present. For a general non-symmetric square matrix the eigenvalues and singular values are unrelated in magnitude beyond the bound that the largest absolute eigenvalue never exceeds `sigma_1`. ## Why this is a bad recipe numerically The relation is a proof device, not an algorithm. Forming A^T A **squares the condition number**: if the smallest singular value is `1e-8` times the largest, then in A^T A that ratio is `1e-16`, at or below the resolution of double-precision arithmetic. The small singular values, which are exactly the ones a rank or stability decision depends on, get lost in rounding error. It also costs an extra matrix product and destroys any sparsity A had. Stable practice works on A directly, applying orthogonal transformations that preserve singular values and never form the cross-product matrix. The takeaway for an interview is to state the identity confidently as mathematics and then say plainly that you would not compute it that way. ## Common mistakes Forgetting the square root and reporting `lambda_i` as a singular value; claiming A^T A can have a negative eigenvalue; and saying both U and V come out of A^T A, when A^T A gives V and A A^T gives U.