What does the singular value decomposition A = U S V^T tell you about a matrix A?

## The statement For **any** real matrix A of shape m-by-n there exist matrices U, S, V such that `A = U S V^T` where U is m-by-m with orthonormal columns (`U^T U = I`), V is n-by-n with orthonormal columns (`V^T V = I`), and S is m-by-n, zero everywhere off the main diagonal, with diagonal entries `sigma_1 >= sigma_2 >= ... >= sigma_p >= 0` for `p = min(m, n)`. Those diagonal entries are the **singular values**; the columns of V are the **right singular vectors** and the columns of U the **left singular vectors**. No condition is attached. The matrix need not be square, symmetric, invertible or full rank. That unconditional existence is the single most useful thing about the SVD. ## What each factor means Read the factorization right to left as what happens to a vector x: 1. `V^T x` re-expresses x in the basis of right singular vectors. Because V is orthogonal this is a pure rotation (possibly with a reflection) — no stretching, no shearing. 2. Multiplying by S scales coordinate i by `sigma_i`, and pads or truncates to reach dimension m. 3. Multiplying by U rotates the result into the output space. So every linear map, however complicated it looks entrywise, is a **rotation, an axis-aligned stretch, and a rotation**. If you feed the unit sphere of inputs through A you get an ellipsoid; the singular values are the lengths of that ellipsoid's semi-axes and the columns of U are the directions those axes point. The pairing is captured by `A v_i = sigma_i u_i`: the i-th right singular vector is sent to the i-th left singular vector, stretched by `sigma_i`. ## What you read off it - **Rank.** The rank of A is exactly the number of nonzero singular values. This is the numerically trustworthy definition of rank: counting how many singular values sit above a sensible tolerance. - **Largest stretch.** `sigma_1 = max over ||x|| = 1 of ||A x||`, attained at `x = v_1`. This is the spectral norm of A. - **Smallest stretch.** For a square invertible A, `sigma_n` is the smallest stretch, and the ratio `sigma_1 / sigma_n` is the condition number, a bound on how badly A amplifies relative error. - **Layers.** Expanding the product gives `A = sigma_1 u_1 v_1^T + sigma_2 u_2 v_2^T + ...`, a sum of rank-1 pieces ordered by importance. Truncating that sum is the whole business of low-rank approximation. ## Full versus thin The **full** SVD keeps U at m-by-m and V at n-by-n, with S padded by zero rows or columns to be m-by-n. The **thin** or compact form keeps only the r columns of U and V that pair with nonzero singular values, giving `A = U_r S_r V_r^T` with S_r an invertible r-by-r diagonal. Both reproduce A exactly; the thin form simply drops the pieces that get multiplied by zero, and it is what is actually stored. ## Uniqueness The singular values themselves are unique. The singular vectors are not, in two ways. First, `u_i` and `v_i` can both be negated together without changing `sigma_i u_i v_i^T`. Second, if a singular value is repeated, any orthonormal rotation inside the corresponding subspace is equally valid. Vectors paired with zero singular values are almost entirely free. This matters when comparing two decompositions of the same matrix: expecting them to agree entry by entry is a mistake, but the singular values must agree. ## Why it is not an eigendecomposition An eigenvector satisfies `A v = lambda v`, which forces input and output to live in the same space, so only square matrices are candidates, and even then some square matrices cannot be diagonalized. The SVD sidesteps both problems by allowing two different orthonormal bases, V on the input side and U on the output side. That extra freedom is exactly what buys unconditional existence. ## Common mistakes Saying a singular value is negative (any sign is absorbed into U or V), claiming rectangular matrices have no SVD, and assuming the singular values of A are its eigenvalues. For a general square matrix the two sets are different quantities entirely.