Working With Data in One Process

A labelled table stores fields, not records: each named column is one run of values in a single representation. A row is not stored as a unit — it is a cross-section of every column, assembled on demand.

Names and per-field types make a rectangle self-describing: every field says what it is, and each field can hold a different kind of value. A rectangle addressed only by position carries one representation for everything and offsets to address it by.

A two-axis table cannot hold three coordinates directly, so it flattens one: repeat store and month down the rows beside a measure column, or fold the measure into the column labels. A holding with a third axis keeps all three.

A row label is a per-row key stored beside the values, so a row can be named rather than counted. Nothing checks it for uniqueness, duplicates are legal, and some tabular designs carry no row identity at all.

A uniform-type rectangle holds one representation for every cell, so the single text field decides it for all of them. The numeric values stop being packed numbers and are re-held in whatever form can also carry text.

Eighty million bytes: the declared width times the row count. That figure is the packed value buffer only — anything stored beside it, such as an absence mask, the held-once values behind codes, or the table's row labels, is extra.

Stating the column's type at construction fixes it as text before any value is stored, so leading zeros survive and long identifiers stay exact. Left undecided, digits are treated as a quantity and both are lost for good.

Every operation has a result rule that fixes what its output column is made of; the inputs only feed it. An average is a division, defined over fractional values, so whole numbers in still gives fractional out.

A column carries one representation for all its rows, so no row can be special. The misfit forces a whole-column choice: fall back to a form that holds anything, refuse the value, or tag every row with its kind.

One reference per row, with the values allocated separately elsewhere. The column's own buffer is a fixed slot per row, so a shallow count stops there; following the references adds every value's bytes plus its per-allocation bookkeeping.

It hands back a newly allocated column of one million products, one per position, and leaves both operands as they were. Your program makes one dispatch, but the million-step loop still runs - inside the library's compiled pass.

A one-value operand has nothing to disagree about: every rule reuses it at every position, giving 1,000 results. Three values against 1,000 positions is a real size conflict, and designs resolve it differently - by refusing, by repeating, or by filling with absent values.

The ordinary whole-column text operation computes a new column and returns it; it does not edit the values where they sit. If nothing captures the result, or it is captured under a name the later check does not read, the table keeps the old values and no error is raised.

Four fixed overheads move from once-per-column to once-per-record: a run-time type decision per value, each value handed back as a wrapped language object, a fresh record built to reach its fields, and a result container reallocated as it grows.

What the surface hands the body. Called once per record, it is the same loop with a crossing into your code at every record. Called once with the whole column, it is one hand-off and the work stays inside compiled code.

The row label still names the same record; the offset does not. A label is an identity the row carries with it, while an offset only describes where the row happens to sit right now.

A range of offsets in a zero-based host is half-open — the endpoint is excluded — so it returns three rows. A range of labels includes both ends, so it returns four. Same-looking request, different rule.

Region sets the overall order and groups the rows into blocks; amount only arranges rows inside a block where the region is equal. Each key carries its own direction, so mixing ascending and descending in one step is normal.

Two condition columns combine position by position into a third full-length column, one outcome per row. The host's scalar connective wants a single truth value from each side, and a whole column has none, so it either refuses or quietly answers a different question.

A condition column is built first: one true-or-false outcome per row, so 1,000 entries long, exactly as long as the table. That full-length column then addresses the table, and the rows whose outcome was true come back whole.

Zero is a measurement whose answer was none, a text value of length zero is a value that was supplied and contains no characters, and only a cell with nothing recorded is absence: no value exists there at all.

They answer different questions. The row count is how many records the table holds; the count of present values is how many of them recorded something in that column. The difference, 1,600, is that column's number of holes.

Filling with a constant stacks 1,500 identical values at one point, and where that point sits decides the damage: a zero fill in a column centred above zero drags the average down and leaves filled cells indistinguishable from measured zeros.

Forty cells of the result hold no value, and the remaining cells hold the sum. An operator meeting an absent operand carries the absence into its output rather than skipping it, substituting a zero, or raising an error.

The count of present values, 8,400 — not the 10,000 rows. An aggregate that steps over the holes removes them first, so the reported number is an average of what was recorded, over a population smaller than the table.

Collapsing to one row per group returns 50 rows, one per distinct key value. Aligning that same summary back onto its rows returns all 10,000, with each row carrying its own group's value beside it.

One running value per customer — the largest amount seen so far, replaced when a bigger one arrives. Each record is folded in and then dropped, so the room used tracks the number of customers, not the number of orders.

Each result row is identified by a pair, one value from each grouping key, and the result holds exactly one row per pair that actually occurs in the data, not one row per pair the two columns could form.

A grouped result carries one row per key value that actually occurred, so the three unlisted categories had no rows in this input. They are absent from the result rather than present with a zero.

Split, apply, combine. The split gives every row a grouping key and records which rows share each key; the apply runs one computation per key; the combine assembles those per-key answers into a result.

The tool decides, not the expression. Designs that carry a per-row identifier pair the two operands by that identifier first; designs that carry none pair strictly by position and refuse operands of different lengths.

Equality compares stored values, not the rendering. Two codes that print alike can differ by letter case, leading or trailing spaces, an invisible character, a lost leading zero, or one side holding digits as text and the other as numbers.

Losing 60 rows means the match kept only rows partnered on both sides. The three alternatives keep the orders table whole, the customer list whole, or both whole, and unpartnered rows then survive with the other side's columns holding the absent-value marker.

Stacking adds records, so the result is longer and the hazard is two pieces whose column sets do not agree. Gluing side by side adds attributes, so the result is wider and the hazard is rows paired in an order nobody checked.

A key match returns, for each key value, the first side's occurrence count multiplied by the second side's. If a customer identifier appears more than once in the reference table, the orders carrying it are copied and the total grows.

One measurement: store and date say which subject and moment, measure names what was counted, amount carries the number. That is the long layout — one row per measurement, with the measure's name in a cell, not in a header.

Each combination of row key and new header must occur exactly once. The result has one slot per combination, so two input rows sharing the same identifying values and the same header value leave one cell with two candidate contents.

Each output column is identified by two name parts, one from each name column, with one column per pair the data holds. Whether those parts stay separately addressable or are composed into a single string depends on the tool's label space.

Three roles: the identifying columns that stay put as the row key, the header source whose distinct values become the new column headers, and the cell source whose entries fill those cells. Folding back reads the same declaration backwards.

The wide layout — one column per measure — puts both measures on the same row, so the comparison is one column-against-column step. In the long layout, one row per measurement, the two numbers sit in different rows and must be brought together first.

One of three things, depending on the design: a row carrying an absent value, a row the same call already filled, or no row at all. Confirm which by comparing the returned row count with the months the span should contain.

Promoting the timestamp makes it what rows are named and ordered by, so a range can be asked for by naming two moments and time operations find the ordering unasked. Only one of the record's times can hold that slot.

A stamp with no zone is a clock reading, not a point on the world timeline: it says what a clock showed, not when. Until the recording zone is attached, it cannot be safely compared with, or converted for, another source.

Match each quote to the most recent price record whose stamp is at or before the quote's stamp - an inexact ordered match rather than an equality match. The driving side sets the row count: at most one matched record each.

A moving window of fixed length keeps its edges a fixed distance apart, so old records drop out as new ones enter. A window anchored at the start never moves its left edge, so each answer covers everything so far.

Every field in that column looked like digits, so the reader chose a numeric representation and parsed the characters into values. Leading zeros carry no numeric value, so they were discarded during the read. Stating the column as text prevents it.

A reader over a file with no declared types guesses each column's type from the rows it inspects, then holds every value in that column to that one representation. Different rows next month can produce a different guess.

One piece is a batch of whole records handed back per turn of the loop, not an arbitrary byte range: cuts must fall on record boundaries, and the line naming the columns belongs to the first turn only.

A self-describing binary columnar file carries its own type declaration, so the read applies it. Delimited text carries characters only, so something outside the file decides what each field means — on every read, and possibly differently the next time.

Compare rows in the table against a count you knew independently: lines in the file minus the line that names the columns, or the count the producer stated. Equal is reassurance; unequal is the alarm the read never raised.

A connecting line claims the space between two marks was travelled: that intermediate positions exist and the value passed through them. Between unordered categories nothing lies in between, so the line asserts a path that does not exist.

A bound colour is a mapping: the column's values decide each mark's colour, and a scale plus a legend let a reader read them back. A fixed colour is decoration — every mark gets it, and it carries no data.

Neither shape is a property of the data alone. The chart counted rows into buckets before drawing, so the picture is of those counts, and the bucket width - which the tool chose if you did not - decides how many peaks appear.

A bar encodes value as length from the baseline, so a baseline of 92 makes each length proportional to value minus 92 rather than to the value: 94 and 98 become lengths of 2 and 6.

Settle which comparison the reader will make: magnitude, change along an ordered dimension, spread of one column, relationship between two, or rank. The comparison picks the mark, and a mark chosen for looks answers a different question than the one asked.

Plan-then-run execution: the three steps only recorded a description of the work, so they returned immediately. The later call is the trigger, where the whole recorded plan finally executes and every byte of work is paid at once.

A total is determined by the records already read, but an exact median is not: a record still unread can move the middle value. Only the total has a combine step that absorbs whatever arrives later.

The in-memory representation of the columns sets the multiple, not the row count. Values held as one runtime object per row, header and reference included, cost several times their text; fixed-width typed columns can cost less than the file did.

Averaging piece means gives every piece one vote while the pieces hold different numbers of rows, so the answer drifts toward whatever the small pieces contained. Carry a running total and a running count instead, and divide once at the end.

Three local savings: columns the plan never references are never read or decoded, a row condition can be applied during the read instead of after it, and consecutive element-wise steps can collapse into one traversal with no intermediate buffer.

A row count is a claim a machine can fail on, while scrolling only samples the rows you happen to see. Counts either side of one step catch rows silently lost or multiplied anywhere in the table.

An interactive session keeps every value any block ever bound, including blocks later edited or deleted. The number on screen reflects the order blocks were actually executed, and the file records neither that order nor the code that produced it.

Record the number of rows after every step in one instrumented run, then read the sequence for the first adjacent pair where it falls. Two endpoint numbers give the size of a loss and never its location.

A column ordered character by character is being held as text rather than as numbers: '10' comes before '9' because '1' comes before '9'. Nothing raised because putting text in order is a legal operation.

Nothing promises that two versions emit rows in the same sequence, so a position-by-position comparison is measuring order rather than correctness. Align both outputs on the columns that identify a row, then compare the matched pairs.