How do you keep column definitions and lineage trustworthy for a published analytics model?

## Why documentation rots, structurally Every data platform has a catalog full of confident, wrong descriptions. The cause is not laziness. It is that the description lives somewhere the code cannot reach, so changing the model has no mechanical consequence for the sentence describing it. Nothing fails. Nobody is told. Every durable fix follows from that diagnosis: move the documentation into the path of change, or make it executable. ## Documentation as code, reviewed with the change When a model's column descriptions live next to its definition in version control, three things follow. The change to the logic and the change to the description appear in the same review, so a reviewer can see the sentence is now false. The history of the definition is inspectable — "when did we start excluding tax from this measure?" is answerable. And ownership is expressed by the same mechanism that governs the code. This is the point at which teams reach for whatever their transformation tooling offers to attach descriptions to models. The mechanism matters far less than the property: the description must be in the diff. ## What actually needs documenting Most catalog entries fail by describing the obvious — `customer_id` documented as "the customer id" — while omitting everything a consumer would get wrong. The high-value items: - **The grain**, in one sentence: what exactly does one row mean? This is the single most useful line in any model's documentation, and its absence causes the most expensive mistakes. - **Units, currency and scale.** Is `revenue` in dollars or cents? Which currency, and converted at which rate, on which date? - **Timezone and date semantics.** Is `order_date` the customer's local date or UTC? Is a monthly measure calendar-month or a 4-4-5 fiscal period? - **The business rule behind a derived column.** `is_active` is meaningless without "had a non-cancelled subscription on the last day of the month". - **What NULL means** in this column, and whether it is distinguishable from zero or from "not applicable". - **Known exclusions.** Test accounts removed, cancelled orders filtered, internal transfers netted — the same list that makes reconciliation interpretable. ```text fct_subscription_revenue grain: one row per subscription per calendar month owner: revenue-analytics mrr_amount: recognised monthly recurring revenue, USD, net of discounts, excludes tax; source billing.invoice_line is_active: subscription had a non-cancelled status on the last day of the month; NULL never occurs exclusions: internal test accounts (account_type = 'INTERNAL') ``` ## Tests are the documentation that cannot drift This is the strongest single idea in the area. Prose says a column contains one of four status codes; an accepted-values assertion says the same thing and *fails on the day it becomes untrue*. Prose says the table is one row per order line; a uniqueness assertion proves it on every load. So the test suite is a machine-checked subset of the documentation, and it is the subset consumers can actually rely on. That reframes test coverage as a documentation question: the properties a consumer most depends on are the properties that should be assertions rather than sentences. Prose is then reserved for what cannot be asserted — intent, business rules, history, caveats. ## Lineage: derive it, never draw it Column- and table-level lineage answers two questions people ask constantly: *where does this number come from*, and *what breaks if I change this*. Both are load-bearing — the first for trust, the second for safe change. A hand-maintained diagram answers both wrongly within a quarter, and a wrong lineage graph is more dangerous than none because impact analysis based on it is confidently incomplete. Derived lineage — parsed from the transformation code, so it cannot disagree with what actually runs — is the only kind worth relying on for impact analysis. The practical limits are worth knowing. Derived lineage is weakest where the SQL is dynamic or generated, where logic sits in an external process, and where it crosses out of the warehouse into BI tools and spreadsheets — which is exactly where the most sensitive consumers live. Closing that last mile usually means integrating the BI layer's own dependency information rather than pretending the graph is complete. ## Ownership makes the rest work A model with no owner has no one to answer "is this the right table?", no one to update the definition when the business rule changes, and no one paged when its tests fail. Naming an owner per published model, and making that ownership visible wherever consumers discover the model, does more for trustworthiness than any tooling choice. The corollary is to keep the published surface small: a hundred models with owners beats a thousand where ownership is nominal. ## How you know it is working The honest signals are behavioural, not coverage percentages. Do analysts ask in chat what a column means, or do they look it up? When a source changes, does the impact analysis come from the lineage graph or from someone's memory? Does the same metric get recomputed differently in three dashboards? A documentation programme measured by percent-of-columns-described optimizes for filling boxes, which is how catalogs become full of "the customer id".