Span, Basis and Nullspace

## The definition Take a matrix and call its columns a1, a2, ..., an, each a vector with m entries. The columns are **linearly independent** if the equation ``` c1*a1 + c2*a2 + ... + cn*an = 0 ``` has only the trivial solution c1 = c2 = ... = cn = 0. If any choice of weights that are *not all zero* produces the zero vector, the columns are **linearly dependent**. That equation is just `Ac = 0` written out, where c is the vector of weights. So the columns are dependent exactly when the matrix has a non-zero vector in its nullspace, and independent exactly when the nullspace contains only the zero vector. ## What dependence means in words If the columns are dependent, you can solve for one of them in terms of the rest. Suppose c3 is non-zero in a vanishing combination; divide through and you get a3 written as a weighted sum of the other columns. In other words, a3 already lies in the **span** of the others - the set of all their weighted combinations. It points in no direction the others could not already reach, so removing it does not shrink the space the columns can build. This is why dependence is often phrased as redundancy. The columns still contain the same *numbers*, but they contain no extra *directions*. ## The worked example A usage table records `hours_weekday`, `hours_weekend`, and a convenience column `hours_total` defined during ingestion as the sum of the first two. Every row satisfies ``` hours_weekday + hours_weekend - hours_total = 0 ``` which is precisely a vanishing combination with weights (1, 1, -1). No matter how many rows the table has, and no matter how varied the numbers are, that identity holds row by row, so the three columns can never be independent. The three of them span a plane, not a three-dimensional space, and the matrix has rank 2 with a one-dimensional nullspace spanned by (1, 1, -1). Notice what did the damage: not the data, but the *definition* of a column. Exact dependence in real datasets almost always comes from a derived column - a total, a percentage that completes to 100, a difference of two other columns, a unit conversion. ## Rank, span and dimension The **span** of a set of columns is every vector you can build as a weighted combination of them. The **rank** of the matrix is the dimension of that span - the number of genuinely independent directions among the columns. If a matrix has n columns and rank r, then r <= n, with equality exactly when the columns are independent. A matrix whose rank is smaller than its column count is called **rank deficient** or **column rank deficient**. An important consequence: a set of dependent columns can be trimmed to an independent subset that spans exactly the same space. Any such minimal spanning subset is a **basis** of the column space, and every basis of that space has the same size, namely the rank. ## Two common traps **Dependence does not require proportionality.** Columns (1, 0), (0, 1) and (1, 1) are dependent, since the third is the sum of the first two, yet no single column is a multiple of any other. Dependence can involve three, four or all of the columns at once. **Dependence is not correlation.** Correlation is a statistical measure of how two centred columns co-vary, and it takes values across a continuous range. Dependence is a yes/no algebraic fact. Two columns correlated at 0.99 are still independent; two columns tied by an exact identity are dependent regardless of what any summary statistic says. ## Why interviewers ask it Linear dependence is the concept behind rank, the nullspace, and the whole question of whether a linear system pins down a unique answer. Candidates who can state the zero-combination test, name the redundancy it implies, and point at a realistic way a dataset acquires a dependent column have the foundation the rest of the topic builds on.