Before a team acts on a chart that aggregated its rows, what must the chart state about the statistics it computed?
answer
- reproducible, not merely documented
- which statistic, over how many rows
- the rule behind every computed mark
- generate the caption from the drawing code
- scope it to charts that leave the author
basics
~20 sA chart that aggregated should state enough for a reader to reproduce the number: which statistic, over how many rows, the bucket or whisker rule it used, and whether a curve and band were fitted, with what span.
solid answer
~50 sThe test worth applying is reproducibility rather than completeness: a reader who disagrees with the picture should be able to recompute it and argue with a specific setting instead of a feeling. That gives a short list - the statistic each mark carries, the row count behind it, the bucket rule for a distribution, the whisker rule for a box, and the span and origin of any fitted curve or band. Where to put it is the real decision. Generating those lines from the code that drew the chart costs engineering once and never rots; a hand-written caption rots immediately; leaving every author on their own tool's defaults gives you neither. Scope it to charts that leave the team, and be honest that no caption fixes a chart whose comparison was the wrong one to make.
go deeper
Learn the habit before the policy: when you show someone a chart, be able to say what one mark stands for and how many records are inside it. Write that into the caption yourself.
Know which settings actually change a picture - the statistic, the bucket rule, the whisker rule, the span of a fitted curve - so you can state the ones that matter instead of documenting everything indiscriminately.
Argue the test: a reader who disagrees should be able to reproduce a mark and challenge a named setting. Show how you would generate those lines from the drawing code rather than relying on authors to remember defaults they never chose.
Own the tradeoff. Decide the scope, who pays for the tooling, which conventions are fixed team-wide, and how you stop a caption becoming a badge of trust. Say plainly which honesty problems this buys you nothing against.
## Why this is a standing cost, not a chart tweak A chart that aggregated is **a claim about a number nobody on the page computed**. Marks were drawn from statistics the interface produced - counts per bucket, a value per category, positions from a sorted column, a fitted curve - and the settings that produced them exist only in code, or nowhere at all if they were defaults. When a decision is taken on such a chart, an organisation is trusting a chain that has no visible links. That makes it a leadership question rather than an authoring one. Any individual author can be careful; whether the whole team's charts can be challenged is a standard somebody has to set, pay for and maintain. ## The short list worth mandating The useful test is not *what could be documented* but **what a disagreeing reader needs in order to recompute the number and argue with a specific setting**. | what the chart states | what it lets a reader do | what its absence costs | |---|---|---| | which statistic each mark carries | recompute one mark and confirm the claim | two readings of one picture, both defensible | | how many rows are behind each mark | weigh a thinly supported category correctly | a category of a dozen records compared with one of millions | | the bucket rule for a distribution | redraw and see whether a mode survives | a decision taken on a mode that another width does not show | | the whisker rule for a box mark | compare two charts, or set a threshold honestly | two reports counting different anomalies and nobody at fault | | the span and origin of a fitted curve or band | judge how much of the shape is the setting | a smooth line read as a measurement | Notice what is not on the list: the author, the date, the source table alone, or a badge saying the chart came from the standard template. Those are provenance for the data; none of them lets a reader reproduce the statistic. ## Three ways to pay for it, cheapest to dearest over time 1. **Generate the line from the code that drew the chart.** The drawing step knows the statistic, the row count and every setting it applied, so it can write them into a subtitle. It is engineering work once and it cannot drift, because the chart and its caption come from the same call. 2. **Fix the team's defaults explicitly and version them.** Decide the whisker convention, the bucket rule and whether a curve may be drawn without its points, record the decision, and let the shared drawing helpers apply it. This shrinks what needs stating at all, because the convention is stable and citable. 3. **Require a hand-written caption on charts that leave the team.** The weakest of the three: it rots the moment a chart is redrawn, and it relies on the author remembering a setting they did not consciously choose. It is still worth having as the floor for anything that goes to a decision meeting. ## What no disclosure fixes The honest limit of this whole discipline is that it makes a claim auditable, not correct: - **The wrong comparison stays wrong.** If the bars answer a question nobody asked, stating the statistic just documents the mismatch precisely. - **The wrong summary for the shape stays wrong.** A per-record mean over a column dominated by a handful of records is still a poor summary when its definition is printed under it. - **A picture nobody re-derives is still unchecked.** Reproducibility is an option a reader has, not an action they take. ## The rule's own failure mode Every disclosure standard creates the impression that a chart carrying the disclosure has been checked. Teams stop asking questions of captioned charts, and the caption becomes a badge rather than an invitation. Two cheap counterweights: require the reviewer of a decision chart to reproduce **one** mark by hand, and keep the caption short enough that people read it. A long provenance block is read exactly as often as no block at all. ## Scoping it so it survives contact Exploration is not reporting. Charts drawn to think with - dozens a day, never leaving one screen - should carry no obligation at all, and a rule that pretends otherwise gets ignored everywhere, including where it mattered. Draw the line at **charts that leave the author**: pasted into a document, shown in a meeting, or attached to a decision. That is a small enough set to enforce and a large enough set to matter, and it keeps the standard about consequence rather than about ceremony.
- Which charts would you exempt from the rule entirely?Anything drawn to think with and never shown to another person - dozens a day on one screen. A rule that covers exploration is ignored everywhere, including where consequences exist. Draw the line at charts that leave the author: pasted into a document, shown in a meeting, or attached to a decision.
- A chart already circulating carries no settings at all, and a decision is due tomorrow - what do you do?Recompute the marks yourself from the source rows and say what you found. Report the statistic and row count you reproduced, note which settings you had to guess, and treat the original as unverified rather than wrong. That is faster than a debate and it produces the missing caption as a by-product.
- Why is the author's name and the date not enough provenance for a chart?They locate who to ask, not how the number was produced. A reader who disagrees still cannot reproduce a mark, cannot redraw at another bucket width, and cannot tell whether a whisker or a fitted curve came from a rule or a default. Provenance for the data is not provenance for the statistic.
saying these in an interview costs you the question
- Mandates a caption on every chart, including throwaway exploration.
- Believes stating the settings makes the comparison itself correct.
- Leaves each author on their own tool's defaults and calls that consistency.
- Treats the author name and date as provenance for a computed statistic.
- Assumes readers will ask for the settings whenever they need them.
- Writes a provenance block so long that nobody reads any of it.