How do the L1, L2 and L-infinity norms of a vector differ?
answer
- three different ways to measure length
- same formula, different exponent p
- sum, square root, or maximum
- city blocks versus a straight line
- try them all on the vector (3, 4)
basics
~20 sAll three measure a vector's size differently. L1 adds the absolute values of the coordinates, L2 is the square root of the sum of squares, and L-infinity is the largest absolute coordinate. For (3, 4) they give 7, 5 and 4.
solid answer
~40 sThey are members of the same family: `||x||_p = (|x_1|^p + ... + |x_n|^p)^(1/p)`. At `p = 1` you get the L1 or Manhattan norm, the sum of absolute coordinates. At `p = 2` you get the L2 or Euclidean norm, the straight-line length. As `p` grows to infinity every term but the largest becomes negligible, so the L-infinity or Chebyshev norm is just the biggest absolute coordinate. For the vector (3, 4) that is 7, 5 and 4 respectively, and the ordering `||x||_inf <= ||x||_2 <= ||x||_1` holds for every vector. The practical difference is how they weight one large deviation against many small ones: L2 squares, so it punishes a single big coordinate hardest; L1 treats every unit of error alike; L-infinity ignores everything except the worst coordinate.
go deeper
Be ready to compute all three norms of a small vector on the spot and to say in one sentence what each one measures. Knowing that L2 needs the square root is the actual screening test here.
Explain the general Lp formula and why the limit as p grows is the maximum coordinate. An interviewer expects you to justify the ordering L-infinity <= L2 <= L1 rather than just assert it.
Show you pick a norm to match the failure you care about: worst-case component drift argues for L-infinity, many small errors argue for L1, and smooth optimization argues for squared L2. Name the consequence, not just the formula.
Own the framing that the norm encodes what the business counts as an error. Be ready to argue when a total-absolute reading beats a squared one for a metric the whole organisation will optimise against.
## What a norm is A norm is a function that turns a vector into one non-negative number meant to represent its length or size. To qualify as a norm it must satisfy three rules. It is zero only for the zero vector. Scaling the vector by a number `c` multiplies the norm by `|c|` (absolute homogeneity). And the norm of a sum never exceeds the sum of the norms, `||u + v|| <= ||u|| + ||v||` (the triangle inequality). Several different functions satisfy all three, and they genuinely disagree about which of two vectors is larger, so picking one is a modelling decision. ## The Lp family For a vector `x` with coordinates `x_1, ..., x_n` and any `p >= 1`: `||x||_p = (|x_1|^p + |x_2|^p + ... + |x_n|^p)^(1/p)` - **p = 1 (L1, Manhattan, taxicab):** `|x_1| + |x_2| + ... + |x_n|`. Every unit of deviation counts the same no matter which coordinate it lives in. - **p = 2 (L2, Euclidean):** `sqrt(x_1^2 + ... + x_n^2)`. The ordinary straight-line length from geometry. Note the square root: the sum of squares alone is the *squared* L2 norm, not the norm. - **p -> infinity (L-infinity, Chebyshev, max norm):** `max(|x_1|, ..., |x_n|)`. As `p` grows, the largest term dominates the sum so completely that the `1/p` root collapses everything else away. ## A worked example Take the point (3, 4) and measure its distance from the origin. - Manhattan (L1): `3 + 4 = 7`. This is the honest reading when you can only move along axis-aligned steps, like walking a grid of streets: no diagonal shortcut exists, so you really do travel 7 blocks. - Euclidean (L2): `sqrt(9 + 16) = sqrt(25) = 5`. The as-the-crow-flies distance. - Chebyshev (L-infinity): `max(3, 4) = 4`. This is the honest reading when one step may change several coordinates at once, like a chess king that moves one square in any direction: it reaches (3, 4) in 4 moves. Three defensible numbers for the same pair of points. Nothing is wrong with any of them; they answer different questions about what counts as movement. ## Ordering and equivalence For any fixed vector the Lp norm is non-increasing in `p`, which gives the chain `||x||_inf <= ||x||_2 <= ||x||_1`. The three coincide only when at most one coordinate is nonzero. Bounds run the other way too: `||x||_1 <= sqrt(n) * ||x||_2` and `||x||_2 <= sqrt(n) * ||x||_inf`, where `n` is the dimension. In finite dimensions all norms are equivalent in this sense, each bounded above and below by a constant multiple of any other, so questions like whether a sequence converges have the same answer under all of them. What differs is the *geometry* at a fixed budget, which is exactly what an optimizer feels. ## Unit vectors and unit balls Dividing a nonzero vector by its norm, `x / ||x||`, produces a unit vector: same direction, norm exactly 1 in that norm. The set of all vectors with norm 1 is the unit sphere, and the set with norm at most 1 is the unit ball. In two dimensions the L1 ball is a diamond with corners on the axes, the L2 ball is the familiar circle, and the L-infinity ball is an axis-aligned square. The L1 diamond sits inside the L2 circle, which sits inside the L-infinity square, which is the same ordering as the norm chain read backwards. ## Near misses that are not norms Setting `p < 1` breaks the triangle inequality, so those functions are not norms even though the formula still evaluates. The so-called L0 norm, which counts how many coordinates are nonzero, is not a norm either: doubling a vector leaves the count unchanged, so `||2x||_0 = ||x||_0` instead of `2||x||_0`, violating homogeneity. The names are entrenched, but an interviewer may check whether you know they are misnomers. ## Choosing between them Use L2 when you want the everyday notion of length, when rotational symmetry matters (L2 is the only Lp norm unchanged by rotating the coordinate system), or when you want a smooth objective, since the *squared* L2 norm is differentiable everywhere. Use L1 when a few large deviations should not dominate, when you want a total-absolute-error reading, or when you want the corner geometry that drives coordinates to exactly zero. Use L-infinity when the requirement is a worst-case guarantee, for example bounding how far any single component of an error vector may drift.
- What happens to the Lp norm of a fixed vector as p grows toward infinity?It decreases monotonically and converges to the largest absolute coordinate. Raising each `|x_i|` to a high power makes the biggest one dwarf the rest, and the outer `1/p` root then returns essentially that term alone. So `||x||_inf = max |x_i|`, and for any `p < q` you have `||x||_q <= ||x||_p`.
- Is the so-called L0 norm actually a norm?No. It counts the number of nonzero coordinates, and that count does not scale with the vector: `||2x||_0` equals `||x||_0` rather than `2||x||_0`, so it fails absolute homogeneity. It is a useful sparsity measure and a legitimate objective, but calling it a norm is a historical misnomer.
- Why is the squared L2 norm used as an objective more often than the L2 norm itself?Because squaring removes the square root, and the result is a smooth quadratic that is differentiable everywhere, including at the origin where the plain L2 norm has a kink. Its gradient is simply `2x`. Since squaring is monotone on non-negative numbers, minimizing the square and minimizing the norm give the same solution.
Measuring a trip across a city: L1 is the blocks you walk on the street grid, L2 is the crow-flies distance a drone covers, and L-infinity is how many diagonal king-moves a chess piece would need.
saying these in an interview costs you the question
- Calls the sum of squares the L2 norm, forgetting the square root
- Assumes all norms rank two vectors in the same order
- Reads L-infinity as an infinite sum rather than a maximum
- Thinks the L0 norm is a genuine norm
- Says Manhattan distance is always longer than Euclidean in every geometry