Two feature columns correlate at 0.999 - is that matrix rank deficient?

## Two different claims **Exact linear dependence** is an algebraic statement: there exist weights, not all zero, with c1*a1 + c2*a2 = 0. For two columns this means one is a scalar multiple of the other. It is a yes/no property with no middle ground, and it is exactly what makes the rank drop. **Correlation** is a statistical summary of two centred columns, taking any value in [-1, 1]. It equals +1 or -1 precisely when one column is an exact affine function of the other, that is, when the centred columns are exactly proportional. Anything strictly inside that range - 0.999 included - means no exact identity holds. So the direct answer is: at 0.999 the columns are linearly independent, and a matrix with those two columns has rank 2, full column rank. Nothing about the algebra is degenerate. ## Why that answer is not the end of it Rank is a **discontinuous** function of a matrix's entries. Perturb one entry of an exactly rank-deficient matrix by 10^-12 and the rank jumps from 2 to 3, even though the matrix is, in every meaningful sense, the same matrix. Conversely a matrix at correlation 0.999 is full rank, yet it is a whisker away from one that is not. A property that flips on an invisible change is a poor instrument for describing measured data, where every entry carries noise anyway. The honest framing is **distance to rank deficiency**: how large a perturbation would it take to make the columns exactly dependent? For exactly proportional columns the distance is zero. At 0.999 the distance is tiny. At correlation 0.2 it is large. That is a continuous quantity with a continuous meaning, which is what real data needs. ## The geometry Two independent columns span a plane. When the columns are nearly proportional, that plane is real but the data barely explores one of its directions: almost all of the variation runs along a single line, with a sliver of spread perpendicular to it. The span is two-dimensional in the algebraic sense while being effectively one-dimensional in the practical sense. Because the second direction is so thinly supported, the weights needed to express any given vector in terms of the two columns become enormous and delicately balanced - a large positive weight on one column nearly cancelled by a large negative weight on the other. Nudge the data and those weights move dramatically, even though the vector they reconstruct barely changes. That sensitivity, not the rank, is what actually causes trouble. ## Exact versus near, in real datasets A useful discriminator is *where the relationship comes from*. - **Exact dependence** almost always comes from a **definition**: a column computed as the sum, difference, ratio or unit conversion of others; a duplicated column under two names; a quantity recorded once in two systems and copied. Exact identities are manufactured, not measured. - **Near-collinearity** almost always comes from **measurement**: two sensors observing the same physical thing, two questionnaire items asking nearly the same question, two metrics driven by the same underlying quantity. Noise makes an exact identity essentially impossible. So when you see a correlation of 0.999, the first move is not to pick a threshold - it is to read the definition of the two columns in the pipeline. If one is derived from the other, you have exact dependence hiding behind rounding, and the fix is structural. If they are two genuine measurements, you have near-collinearity, and the question becomes how much independent information the second one really contributes. ## Floating point blurs the line further In finite precision, even an exactly dependent pair of columns rarely produces an exact zero when you test the identity: rounding leaves residue at the level of machine precision. Conversely a near-dependent pair produces residue barely larger than that. Any procedure that decides rank by computing whether something equals zero is therefore making a threshold judgement, whether or not it admits to it. This is why 'what is the rank of this data matrix?' is not really a well-posed question for measured data; 'how close is it to rank deficient, and does that closeness matter for what I am computing?' is. ## What a strong answer sounds like State the algebra first - 0.999 is not 1, so the columns are independent and the matrix is full rank. Then immediately explain why the algebra is not the point: rank is a knife-edge property, near-dependence is continuous, and the practical consequence is extreme sensitivity of any quantity derived from reconstructing one column from the others. Finish by distinguishing derived columns (exact dependence, a definitional problem) from co-varying measurements (near-collinearity, a data problem). Candidates who only say 'yes, it is basically singular' have skipped the distinction the question is testing.