skip to content

Why don't the counts from status = 'active' and status <> 'active' add up to the table total?

level: seniorimportance: should knowfreq 47%

answer

  1. The two filters both reject the same rows
  2. Negation does not recover them
  3. Count the rows in neither bucket
  4. One decisive branch fixes the disjunction
  5. Or make absence its own visible category

basics

~10 s

Rows with a NULL status make both predicates UNKNOWN, and WHERE keeps only TRUE, so those rows fall out of both buckets. A predicate and its negation are not exhaustive over a nullable column.

solid answer

~50 s

Both filters are comparisons, so on a row where `status` is NULL each evaluates to UNKNOWN, and `WHERE` discards UNKNOWN exactly as it discards FALSE. The NULL rows therefore appear in neither result and the two counts fall short of the table total by precisely the number of NULLs. Wrapping the second filter in `NOT` does not help — `NOT UNKNOWN` is UNKNOWN. To make the buckets exhaustive you must handle absence explicitly: `WHERE status <> 'active' OR status IS NULL`, or the NULL-safe `WHERE status IS DISTINCT FROM 'active'`, which returns TRUE when the values differ *or* when exactly one side is NULL. Diagnose it by running `SELECT COUNT(*) FROM t WHERE status IS NULL` and checking that it accounts for the gap; the deeper question is then whether the column should be nullable at all.

code

sql · 6 lines
sql
-- 6 + 3 = 9, but the table holds 10 rows
SELECT COUNT(*) FROM subscriptions WHERE status = 'active';
SELECT COUNT(*) FROM subscriptions WHERE status <> 'active';

-- the missing row
SELECT COUNT(*) FROM subscriptions WHERE status IS NULL;

go deeper

for a junior

Know that a NULL row satisfies neither col = 'x' nor col <> 'x', so the two filters do not cover the table. Adding OR col IS NULL is the fix to remember.

for a middle

Explain the mechanism: both comparisons yield UNKNOWN, WHERE keeps only TRUE, and NOT leaves UNKNOWN unchanged. Show the reconciling count and both rewrites, including the NULL-safe comparison.

for a senior

Diagnose from the symptom — buckets not summing to the total — confirm with a NULL count, and then sweep every other consumer of that column rather than patching the one reported query.

for a principal

Own the modelling call: an explicit 'unknown' member on a NOT NULL column removes this failure mode from every future query, and a reconciliation assertion in the pipeline stops it from recurring silently.

## The symptom A dashboard shows an "active" tile and a "not active" tile, and their sum is smaller than the row count of the table. Nobody deleted anything; the two queries look like exact opposites: ```sql SELECT COUNT(*) FROM subscriptions WHERE status = 'active'; -- 6 SELECT COUNT(*) FROM subscriptions WHERE status <> 'active'; -- 3 SELECT COUNT(*) FROM subscriptions; -- 10 ``` One row is missing from the breakdown. It is the row whose `status` is NULL. ## Why both filters reject the same row On a row where `status` is NULL, `status = 'active'` evaluates to UNKNOWN — SQL cannot decide whether an absent value equals `'active'`. Crucially, `status <> 'active'` is *also* UNKNOWN, for the same reason: not knowing what the value is means not knowing that it differs either. `WHERE` retains a row only when the condition is TRUE, so both queries discard it. The usual instinct — write the second filter as a negation — changes nothing: ```sql SELECT COUNT(*) FROM subscriptions WHERE NOT (status = 'active'); -- still 3 ``` because `NOT UNKNOWN` is UNKNOWN. In three-valued logic a predicate and its negation are not complementary: they partition only the rows where the predicate is *decidable*. ## Diagnosing it The check is one query: ```sql SELECT COUNT(*) AS total, COUNT(*) FILTER (WHERE status IS NULL) AS unknown_status FROM subscriptions; ``` (or a plain `WHERE status IS NULL` count on engines without `FILTER`). If `unknown_status` equals the gap between the total and the sum of the buckets, the diagnosis is confirmed. This should be an instinct: whenever category counts fail to reconcile to a total, suspect a nullable categorising column before suspecting the data. The same failure shows up in subtler places — a job that processes `WHERE processed_at IS NULL` and a monitor that counts `WHERE processed_at IS NOT NULL` reconcile fine, because `IS NULL` predicates are two-valued; but a job split on `WHERE priority > 5` and `WHERE priority <= 5` silently leaves NULL-priority rows unprocessed by *either* branch. That class of bug — work that no worker claims — is much more expensive than a wrong dashboard number. ## Making the buckets exhaustive Three correct shapes, in increasing order of elegance: ```sql -- 1. explicit extra branch, portable everywhere WHERE status <> 'active' OR status IS NULL -- 2. NULL-safe comparison (standard; check engine support) WHERE status IS DISTINCT FROM 'active' -- 3. three explicit buckets rather than two SELECT COALESCE(status, '(unknown)') AS bucket, COUNT(*) FROM subscriptions GROUP BY COALESCE(status, '(unknown)'); ``` Option 1 works because `IS NULL` is two-valued: it contributes a decisive TRUE that the surrounding `OR` absorbs. Option 2 says the same thing in one predicate — `IS DISTINCT FROM` is TRUE when the values differ *or* when exactly one side is NULL, and never returns UNKNOWN. Option 3 is often the best answer for a report: instead of pretending the world is binary, it surfaces the missing-value population as its own visible category, so the next person to read the dashboard sees the gap rather than losing it. ## The judgment beyond the fix The interesting part of this question at senior level is not the rewrite; it is what you do next. - **Decide what the NULL means.** "Not yet set", "not applicable" and "we lost the value in a migration" call for different treatments. Bucketing them all as `(unknown)` is honest; folding them into `'inactive'` is a claim about the domain that may be false. - **Ask whether the column should be nullable.** A status column that always has a real value in the domain is better expressed as `NOT NULL` with an explicit `'unknown'` member, which removes three-valued reasoning from every query that will ever touch it. That trade — an extra enum member versus a nullable column — is the durable fix. - **Fix every consumer, not just the failing one.** If one dashboard tile was wrong, every other query filtering that column with `=` or `<>` is suspect. Grep for them. - **Add a reconciliation check.** A test or monitor asserting that the buckets sum to the total catches the regression the next time someone adds a category. ## How to answer Lead with the mechanism in one sentence — UNKNOWN is discarded by `WHERE`, so predicate and negation are not exhaustive — then the diagnostic count, then the rewrite, then the modelling question about whether the column should be nullable at all.

  • Would rewriting the second filter as NOT (status = 'active') fix the gap?
    No. On a NULL row the inner comparison is UNKNOWN, and `NOT UNKNOWN` is UNKNOWN, so `WHERE` discards the row exactly as before. Negation cannot convert "cannot tell" into a decision; only a two-valued predicate such as `IS NULL` or `IS DISTINCT FROM` can bring those rows back.
  • What is the durable fix rather than the query-level one?
    Decide whether the column should be nullable at all. If every row has a real status in the domain, declare it `NOT NULL` with an explicit `'unknown'` member: that removes three-valued reasoning from every query that will ever read the column, instead of relying on each author to remember an extra OR branch.
  • Where does this failure hurt more than in a dashboard?
    In work-splitting. If one job takes `WHERE priority > 5` and another takes `WHERE priority <= 5`, rows with a NULL priority are claimed by neither and are never processed — silently, with no error and no backlog alarm. Reconciling the branch counts against the total is the cheap monitor that catches it.

saying these in an interview costs you the question

  • Assuming a predicate and its negation partition the table
  • Thinking NOT recovers the rows the comparison lost
  • Blaming missing data or a bad join before checking for NULLs
  • Folding NULL rows into the 'inactive' bucket without asking what NULL means
  • Fixing only the query that was reported and leaving sibling filters wrong

context