In Great Expectations, what does the mostly parameter change about an expectation's result?

## The default is unforgiving, on purpose A column map expectation in Great Expectations evaluates each row of a column and counts how many fail. The default verdict is binary and strict: one unexpected row and `success` is false. That is the right default for genuine invariants - a primary key that is null once is a broken key. But plenty of real columns are not invariants. A free-text `email` field captured from a marketing form will contain garbage at some small rate forever; a `phone` column will hold a handful of unparseable entries; an enrichment field is null for the tiny fraction of rows the vendor could not match. If those columns fail your gate every night, the gate stops meaning anything, and people start rerunning past it out of habit. That is what `mostly` exists to prevent. ## What mostly actually does `mostly` takes a fraction in the range 0 to 1 and converts the verdict into a threshold test. With `mostly=0.99`, the expectation succeeds when the proportion of evaluated rows meeting the condition is at least 0.99. Crucially it changes **only the verdict**. The result block still records `element_count`, `unexpected_count`, `unexpected_percent` and a sample of the actual offending values, so the failing rows remain visible in the validation result and in Data Docs even on a green run. You are declaring an acceptable rate, not blinding the check. ## Where it applies and where it does not It is meaningful only where there is a per-row notion of success - the `expect_column_values_to_*` family: not-null, unique, between, in-set, match-regex, match-strftime-format. It has no meaning on: - **table expectations** like `expect_table_row_count_to_be_between` or `expect_table_columns_to_match_ordered_list`, which make one assertion about the batch; - **column aggregate expectations** like `expect_column_mean_to_be_between`, which reduce the column to a single statistic before comparing. If you find yourself wanting a tolerance on an aggregate, what you actually want is a wider `min_value`/`max_value` band. ## The interaction that surprises people The denominator matters. For most map expectations other than the explicit null checks, missing values are excluded from evaluation - so `mostly` is a fraction of the **non-null** rows, not of the batch. A column that is 90% null and whose remaining 10% is clean will sail past `expect_column_values_to_match_regex(..., mostly=0.99)`, because the nulls were never in the denominator. The fix is to state completeness separately: pair the format expectation with `expect_column_values_to_not_be_null`, itself given a `mostly` if some nullness is legitimate. Two expectations, two thresholds, two clear failure messages. ## The operational hazard: slow drift A threshold converts a continuous signal into a binary one, and that is exactly what makes it dangerous over time. Suppose a column normally runs 0.3% unparseable and you set `mostly=0.99`. An upstream form change pushes it to 0.9%: a threefold degradation, and your gate stays green for months until a bad day tips it over 1% and you get a page with no history to explain it. Two habits mitigate this. First, set the threshold close to observed reality rather than at a round comfortable number, so the headroom is small and deliberate. Second, treat `unexpected_percent` as a metric you look at over time, not merely as an input to a boolean - the trend is where the early warning lives. ## Choosing the number Derive it, do not guess it. Measure the current rate over a few weeks of batches, ask whether that rate is *acceptable* or merely *current*, and set the threshold just above the acceptable rate. If the honest answer is that any bad row is a defect, do not use `mostly` at all - use the strict default and route the rejects, so the pipeline can proceed while the offending rows are captured for someone to fix. A tolerance that is really a resignation should be written down as such, with an owner and a date, not buried as a float in a suite. ## In an interview Say what it does in one line, then show judgment: it is a declared acceptable defect rate, the result still reports the true counts, it applies to row-wise expectations only, its denominator excludes nulls for most expectations, and its cost is that it masks gradual degradation unless someone watches the rate.