What is the difference between code-execution coverage and requirement coverage?

juniormust knowfreq 62%

answer

Two ratios, two different denominators
One divides by code, one by agreements
Unbuilt behaviour leaves no unrun code
Unrequested code has no requirement term
Name the denominator every time

basics

~20 s

Code-execution coverage measures how much of the written code ran during testing; requirement coverage measures how many agreed requirements have a check confirming them. The denominators differ, so one can be high while the other is low.

solid answer

~50 s

They are two ratios over two different denominators. Code-execution coverage divides by the code that exists: what share of it ran at least once while the tests ran. Requirement coverage divides by the agreed list of behaviour: what share of requirements, or of the acceptance criteria beneath them, has at least one check confirming it. Nothing keeps the two lists in step. Behaviour that was agreed but never built adds nothing to the code denominator, so it can only show on the requirement side; code nobody requested — retry paths, defensive branches, leftovers of a removed feature — has no term on the requirement side at all. Ask the code-side figure when you want to know what exists but is never exercised, and the requirement-side figure when you want to know whether what was asked for has been confirmed.

They are two ratios over two different denominators. **Code-execution coverage** divides by the code that exists: what share of it ran at least once while the tests ran. **Requirement coverage** divides by the agreed list of behaviour: what share of requirements, or of the acceptance criteria beneath them, has at least one check confirming it. Nothing keeps the two lists in step. Behaviour that was agreed but never built adds nothing to the code denominator, so it can only show on the requirement side; code nobody requested — retry paths, defensive branches, leftovers of a removed feature — has no term on the requirement side at all. Ask the code-side figure when you want to know what exists but is never exercised, and the requirement-side figure when you want to know whether what was asked for has been confirmed.

code

pseudocode · 6 lines

pseudocode

executedRatio    = unitsOfCodeExercised / unitsOfCodeWritten
verifiedRatio    = requirementsWithPassingCheck / requirementsAgreed

// same release, same suite, two unrelated denominators
report("code exercised",        executedRatio)   // 0.94
report("requirements confirmed", verifiedRatio)  // 0.61

go deeper

for a junior

Be ready to say, in one sentence each, what the two figures divide by: the code that exists, and the list of agreed behaviour. Knowing that they are different denominators is the whole of the junior expectation here.

for a middle

Explain the mechanics of the divergence: agreed-but-unbuilt behaviour leaves no unexecuted code, and unrequested code has no term in the requirement list. Give one concrete example in each direction.

for a senior

Show that you pick the figure by the question in front of you — the requirement side for a scope or sign-off conversation, the code side before a refactor — and that you always quote a denominator alongside a number.

for a principal

Own the point that both are inventories of existence rather than evidence of confidence, and be ready to say what a team should measure or argue about once both figures are high and the risk is still invisible.

Code-execution coverage is the share of the code that exists which ran at least once while the tests ran. Its denominator is the codebase as written: every unit of code somebody committed is in it, whether or not anyone ever asked for that code.

Requirement coverage is the share of agreed requirements — or of the acceptance criteria beneath them — that have at least one check linked to them and verifying them. Its denominator is the agreed list of wanted behaviour: everything somebody wrote down, whether or not anyone built it.

Why the two move independently

The two lists are assembled by different people at different moments. The code list grows whenever an engineer commits something, including code nobody requested. The requirement list grows whenever a behaviour is agreed, including behaviour nobody has implemented yet. Nothing keeps them in step, so each figure can be near its ceiling while the other sits near its floor.

Four asymmetries produce almost every real case:

Agreed but not built. There is no code for the missing behaviour, so it cannot show up as unexecuted code. It appears only on the requirement side, as a requirement nothing verifies.
Built but never agreed. Retry paths, defensive branches, operator tooling and leftovers of a removed feature all sit in the code denominator and have no term at all in the requirement one.
One shallow check per requirement. Every requirement is marked verified while the check walks only the simplest route through a large implementation, leaving most of that code unexecuted.
Broad exercise, no stated target. A suite can drive most of the code without anybody being able to say which agreed behaviour each run confirms.

Situation	Code-executed figure	Requirement-verified figure
Requirement agreed, never implemented	unchanged — no code to leave unrun	falls: nothing verifies it
Behaviour built with no requirement behind it	falls if never exercised	unchanged — not in its denominator
One easy check per requirement	low	high
Removed feature's code left in place	falls	unchanged
Deep checks written from the implementation	high	high, and possibly wrong

Which number is the right question

Pick the figure by the question you actually have:

"Did we build and confirm what we agreed?" — the requirement side. This is the number for a scope review, a feature-complete conversation, or a release recommendation, because its denominator is what the business asked for.
"Is there code here nobody exercises?" — the code side. This is the number before a large refactor, when sizing risk in a long-lived codebase, or when deciding what is safe to delete, because its denominator is what actually exists to break.
"Are we confident?" — neither on its own. Both are inventories of existence: a requirement with one check counts as verified, and a unit of code reached once counts as run. They tell you what has been touched, not how well.

The pair is more informative than either alone. High on the code side and low on the requirement side usually means the suite was written from the implementation rather than from the agreement. Low on the code side and high on the requirement side usually means the system carries a lot of behaviour nobody wrote down, or that each requirement is confirmed by a single easy route. Low on both is honest and simply unfinished. High on both is the only combination worth arguing about, because it is the one where the remaining risk is invisible to both denominators.

What neither denominator contains

Both lists are written by humans, so both inherit the same blind spot: anything nobody stated and nobody coded is in neither. Unspoken expectations — how the system behaves under an unusual sequence of operations, what happens when a dependency is slow, what an operator sees at three in the morning — fail no requirement, because no requirement exists, and leave no unexecuted code, because no code exists. That is why a candidate who answers "both figures were green" is describing two inventories, not an argument that the release is safe.

The practical habit is to name the denominator every single time. Saying "ninety-four per cent" is meaningless in a mixed room; saying "ninety-four per cent of the code ran, and sixty-one per cent of the agreed requirements have something confirming them" ends the confusion in one sentence and puts the interesting gap — the thirty-nine per cent — where people can see it.

Everyday conversation calls both of these numbers "coverage", which is why two people can agree that coverage is high and still mean unrelated things. They are two ratios, and each one divides by a different list. The denominator is the whole story. **Code-execution coverage** is the share of the code that exists which ran at least once while the tests ran. Its denominator is the codebase as written: every unit of code somebody committed is in it, whether or not anyone ever asked for that code. **Requirement coverage** is the share of agreed requirements — or of the acceptance criteria beneath them — that have at least one check linked to them and verifying them. Its denominator is the agreed list of wanted behaviour: everything somebody wrote down, whether or not anyone built it. ## Why the two move independently The two lists are assembled by different people at different moments. The code list grows whenever an engineer commits something, including code nobody requested. The requirement list grows whenever a behaviour is agreed, including behaviour nobody has implemented yet. Nothing keeps them in step, so each figure can be near its ceiling while the other sits near its floor. Four asymmetries produce almost every real case: 1. **Agreed but not built.** There is no code for the missing behaviour, so it cannot show up as unexecuted code. It appears only on the requirement side, as a requirement nothing verifies. 2. **Built but never agreed.** Retry paths, defensive branches, operator tooling and leftovers of a removed feature all sit in the code denominator and have no term at all in the requirement one. 3. **One shallow check per requirement.** Every requirement is marked verified while the check walks only the simplest route through a large implementation, leaving most of that code unexecuted. 4. **Broad exercise, no stated target.** A suite can drive most of the code without anybody being able to say which agreed behaviour each run confirms. | Situation | Code-executed figure | Requirement-verified figure | | --- | --- | --- | | Requirement agreed, never implemented | unchanged — no code to leave unrun | falls: nothing verifies it | | Behaviour built with no requirement behind it | falls if never exercised | unchanged — not in its denominator | | One easy check per requirement | low | high | | Removed feature's code left in place | falls | unchanged | | Deep checks written from the implementation | high | high, and possibly wrong | ## Which number is the right question Pick the figure by the question you actually have: - **"Did we build and confirm what we agreed?"** — the requirement side. This is the number for a scope review, a feature-complete conversation, or a release recommendation, because its denominator is what the business asked for. - **"Is there code here nobody exercises?"** — the code side. This is the number before a large refactor, when sizing risk in a long-lived codebase, or when deciding what is safe to delete, because its denominator is what actually exists to break. - **"Are we confident?"** — neither on its own. Both are inventories of existence: a requirement with one check counts as verified, and a unit of code reached once counts as run. They tell you what has been touched, not how well. The pair is more informative than either alone. High on the code side and low on the requirement side usually means the suite was written from the implementation rather than from the agreement. Low on the code side and high on the requirement side usually means the system carries a lot of behaviour nobody wrote down, or that each requirement is confirmed by a single easy route. Low on both is honest and simply unfinished. High on both is the only combination worth arguing about, because it is the one where the remaining risk is invisible to both denominators. ## What neither denominator contains Both lists are written by humans, so both inherit the same blind spot: anything nobody stated and nobody coded is in neither. Unspoken expectations — how the system behaves under an unusual sequence of operations, what happens when a dependency is slow, what an operator sees at three in the morning — fail no requirement, because no requirement exists, and leave no unexecuted code, because no code exists. That is why a candidate who answers "both figures were green" is describing two inventories, not an argument that the release is safe. The practical habit is to name the denominator every single time. Saying "ninety-four per cent" is meaningless in a mixed room; saying "ninety-four per cent of the code ran, and sixty-one per cent of the agreed requirements have something confirming them" ends the confusion in one sentence and puts the interesting gap — the thirty-nine per cent — where people can see it.

Can both figures be high and the product still be wrong?
Yes. Both denominators are lists a human wrote. Behaviour nobody stated and nobody coded — how the system behaves when a dependency is slow, what an operator sees during an incident — is in neither list, so it breaks no requirement and leaves no unexecuted code. Both figures are inventories of what has been touched, not arguments that the release is safe.
Which figure would you quote when a product owner asks whether a feature is done?
The requirement-side one, because its denominator is what the business agreed to. It answers whether each stated behaviour has something confirming it. The code-side figure cannot answer that question at all — a behaviour that was never implemented contributes no code, so it never appears as unexecuted code.
Why is quoting a bare percentage in a mixed meeting a problem?
Because two people will hear different denominators. Engineers usually hear the share of code that ran; product and delivery people usually hear the share of agreed behaviour confirmed. Saying which list the number divides by takes one clause and prevents a whole meeting of talking past each other.

Counting how much of a recipe you cooked is not the same as counting how many of the dishes the guests actually ordered.

saying these in an interview costs you the question

Treats one percentage as answering both questions at once
Says high code execution implies every requirement was built
Cannot say what either figure divides by
Thinks requirement coverage counts code rather than agreed behaviours
Assumes the two figures must rise and fall together

What is the difference between code-execution coverage and requirement coverage?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Why the two move independently

Which number is the right question

What neither denominator contains

▾common follow-ups3 questions

▾analogy~10 s

▾red flags5 flags

Why the two move independently

Which number is the right question

What neither denominator contains

answer

go deeper