Element-Wise as the Default Unit

## What the expression hands back Saying an operation once for a whole column instead of once per row - working **column-wise** - produces **a new column**, one result per position, from a single call in your program. Three separate facts live in that sentence, and an interviewer is usually checking all three. - **A column, not a number.** An element-wise operator is defined per position: the value at offset *i* of the left operand combines with the value at offset *i* of the right, and the result at offset *i* is that combination. A million pairs in, a million results out. An operation that folds a column down to a single value is a different kind of operation with a different cost model entirely. - **A fresh allocation.** The plain operator form writes nowhere that already exists. It asks for a buffer the size of the result and fills it, so a million products over eight-byte values cost roughly eight megabytes of new memory, and both operands still hold exactly what they held. - **One dispatch from your program.** Your program executed one statement. What it did *not* do is decide, a million separate times, what kind of things it was multiplying and which multiplication applied. ## The loop did not vanish - it moved This is the part candidates most often get wrong, and it is worth being exact. There is still a loop, and it still runs a million times. What changed is **where** it runs and **what one step of it costs**. In a loop you write in your own program, each step fetches a value stored as **a boxed value** - a full language object with its own header and a pointer to it - works out what type it holds, finds the multiplication for that type, does the arithmetic, allocates a new object for the answer and stores a pointer to it. In **a whole-column operator** - an operation the library provides that takes a whole column and loops over it inside compiled code - the column's single **stored representation** is known before the loop starts: every value shares one representation and one fixed width. The compiled loop therefore reads raw bytes at a fixed stride, multiplies, writes raw bytes, and does nothing else per step. | | A loop you write over records | One whole-column expression | |---|---|---| | Dispatches your program makes | one per value | one in total | | Type decision | once per value | once, before the loop starts | | Value as the arithmetic sees it | a boxed language object | raw bytes of a known width | | Where the million steps run | in your program | in the library's compiled pass | | Result | assembled value by value | one buffer allocated up front | ## Two things it does not do 1. **It does not make the work disappear.** A million multiplications still happen. If you need an answer for a million rows you pay for a million multiplications either way; what you stop paying for is a million decisions about *how* to multiply. 2. **It does not, by itself, use more than one core.** Designs built on a multi-threaded execution engine do split a column across threads. The classic packed-buffer designs run a single-threaded compiled loop and still beat the record loop by a wide margin. Credit the win to removing per-value dispatch first, and treat threading as a second, separate multiplier that some designs offer and others do not. ## The exception to "the operands are untouched" The plain operator allocates and leaves its operands alone, which is what makes an expression safe to write in the middle of a longer computation. Two exceptions are worth carrying: - the **in-place** operator forms write into a buffer that already exists, so anything else holding a reference to that buffer sees the change; - a result that shares an operand's memory rather than owning its own will carry a later write back into the original. If you are relying on an operand still holding its old values three lines later, use the plain form and accept the allocation. ## How to sanity-check the claim yourself 1. Time both forms on the same data at a few sizes - a thousand values, a hundred thousand, ten million - and look at the *ratio*, not the absolute times. If the ratio grows with length, the per-value cost is what you removed. 2. Check the length of what came back. If it is one number, you wrote a fold, not an element-wise expression. 3. Print an operand after the expression. If it changed, you used an in-place form, not the plain one. ## What to say in an interview Name the three facts - a new column, one result per position, one dispatch from your program - then immediately say that the loop moved rather than vanished, and say what one step of it now costs. That last sentence is what separates a candidate reciting a slogan from one who knows the mechanism.