skip to content

A team's JUnit 5 tests iterate over a Java enum but exclude two constants by name so the suite stays green. Six months later a new constant is added and nothing fails. What went wrong, and how would you set up enum-driven tests so a newly added constant cannot slip through untested?

level: seniorimportance: should knowfreq 28%

answer

  1. filtered @EnumSource = frozen coverage
  2. INCLUDE freezes; regex silently drifts
  3. keep one unfiltered contract test
  4. assertion must fail for unhandled constant
  5. pair constant with expectation, no switch in test

basics

~20 s

Excluding by name means new constants are automatically included in the happy-path run but nothing forces anyone to think about them; if the excluded constants are the only ones asserted elsewhere, coverage silently drifts. Prefer exhaustive @EnumSource with no filtering, and pair the constant with its expectation so an unmapped constant fails.

solid answer

~60 s

The exclusion itself is not the bug — the bug is that no test is *exhaustive* over the enum, so adding a constant changes behaviour without changing any assertion. What actually happens with `@EnumSource(mode = EXCLUDE, names = {...})`: the new constant *is* included in that run, but the test only asserts a generic property ('does not throw'), so it passes vacuously. Meanwhile the interesting per-constant behaviour lives in hand-written cases nobody extended. How I'd set it up: - Keep one **exhaustive** `@EnumSource` test with no `names` filter, asserting the contract every constant must satisfy — a mapping exists, a handler is registered, a label is non-null. Unmapped constants then fail on arrival. - Prefer `INCLUDE`/`EXCLUDE` over `MATCH_*` regex modes: literal names are validated against the enum, so renames fail loudly, whereas a regex silently stops matching. - Where each constant has a distinct expected result, carry the expectation *with* the constant in the data rather than a `switch` inside the test. The design rule: adding a constant should be a compile error or a red test, never a silent pass.

code

java · 16 lines
java
// drifts: new constants are never exercised
@ParameterizedTest
@EnumSource(names = {"OPEN", "CLOSED"})
void frozen(Status status) { }

// fails the day a constant is added without a handler
@ParameterizedTest
@EnumSource
void everyStatusHasAHandler(Status status) {
    assertNotNull(registry.handlerFor(status), "no handler for " + status);
}

// renames fail loudly: EXCLUDE validates names against the enum
@ParameterizedTest
@EnumSource(mode = EnumSource.Mode.EXCLUDE, names = "ARCHIVED")
void editableStatuses(Status status) { }

go deeper

for a junior

Recognise that a names filter means new constants are not covered, and that an unfiltered @EnumSource covers all of them.

for a middle

Distinguish the modes: INCLUDE freezes coverage, EXCLUDE auto-includes new constants, regex modes are unvalidated; know that INCLUDE/EXCLUDE names are checked against the enum.

for a senior

Diagnose the weak assertion as the root cause and design for failure-on-arrival: one unfiltered contract test, production code that throws on unknown constants, expectations carried in the data.

for a principal

Frame it as a coverage contract with a closed type — decide what must be exhaustive (invariants) versus explicit (behaviours), and make growing the enum a build-visible event rather than a silent one.

## The failure mode Enums are closed sets, and tests over them are implicitly a claim about *the whole set*. The moment a test filters the set with `names`, that claim narrows — and nothing in the type system or the build records the narrowing. Six months later a constant is added by someone who never saw the test, and the suite is still green because no assertion depended on completeness. There are two distinct variants worth separating in the answer: 1. **`EXCLUDE` narrowing.** `@EnumSource(mode = EXCLUDE, names = {"LEGACY", "UNKNOWN"})` does include a newly added constant automatically. The silent-drift risk here is not exclusion but *weak assertions*: if the shared test only checks 'does not throw', a new constant with no mapping and a default `else` branch passes. Exclusion also rots — the excluded names were excluded for a reason nobody wrote down, and by the time the reason is gone the exclusion stays. 2. **`INCLUDE` narrowing.** `@EnumSource(names = {"OPEN", "CLOSED"})` is worse: a new constant is never exercised at all. INCLUDE lists are how enum tests quietly freeze at the state of the enum on the day they were written. A third, sharper variant: `MATCH_ALL`/`MATCH_ANY` regex modes. These select by pattern against `name()` and are validated against nothing. A rename that breaks the pattern, or a new constant that does not match the naming convention, changes coverage without any signal. By contrast, `INCLUDE`/`EXCLUDE` names *are* validated against the enum's declared constants, so a rename fails the build with a configuration error — a small but real safety property, and the main reason to prefer literal modes for small sets. ## Designing for exhaustiveness The goal is that adding a constant produces a failure somewhere. Three complementary techniques: **1. One unfiltered contract test.** Keep at least one `@EnumSource` with no `names`, asserting the invariant that must hold for every constant: ```java @ParameterizedTest @EnumSource void everyStatusHasAHandler(Status status) { assertNotNull(registry.handlerFor(status), "no handler registered for " + status); } ``` This is the single highest-value enum test in most codebases: it converts 'someone forgot to extend the switch' from a production NPE into a red build. The assertion must be strong enough to actually fail for an unhandled constant — which usually means the production code should *throw* on an unknown constant rather than fall through a permissive `default`. **2. Data that pairs constant with expectation.** When each constant has a different expected result, do not put a `switch` in the test body — that just re-implements the production mapping and can drift in the same direction. Supply pairs from a source that yields argument tuples, and assert that the set of pairs covers the enum (`assertEquals(Status.values().length, cases.size())` in a companion test). **3. Filters carry a reason and an expiry.** If a constant genuinely must be excluded, record why in the test's display name or a comment, and prefer excluding the *behaviour* (a separate test asserting the exception path for that constant) over excluding the case entirely. An excluded constant with no test anywhere is the actual defect. ## Judgement, not dogma Exhaustive enum tests are not free. Over a large enum they multiply invocation counts and can encourage weak assertions purely to keep the shared test green — precisely the failure this question is about. The useful line: exhaustive over *invariants* (every constant maps to something, serialises, has a label), explicit and non-exhaustive over *behaviours* (this constant transitions to that state). And 'the enum grew and no test noticed' is a review-time smell worth naming, not just a runtime one. ## How to answer Diagnose first — the exclusion is a symptom; the missing exhaustive assertion is the cause. Then give the concrete setup: one unfiltered contract test with an assertion strong enough to fail, literal `INCLUDE`/`EXCLUDE` over regex so renames are loud, and expectations carried in the data rather than a `switch` in the test.

  • Why do you prefer INCLUDE/EXCLUDE over the MATCH_ALL and MATCH_ANY regex modes for this?
    INCLUDE and EXCLUDE names are validated against the enum's declared constants, so a typo or a renamed constant fails the test with a configuration error. Regex modes are matched against name() and validated against nothing, so a rename or a new constant that breaks the naming convention silently changes which cases run — the exact drift you are trying to prevent.
  • An exhaustive @EnumSource test exists but a new constant still passes it. What is wrong?
    The assertion is too weak — typically the production code has a permissive default branch, so an unmapped constant returns a fallback instead of failing. Strengthen the production side to throw on an unknown constant, and assert the specific invariant (a handler exists, a label is non-null) rather than merely that nothing was thrown.

saying these in an interview costs you the question

  • Blaming EXCLUDE alone, when the real cause is an assertion too weak to fail for a new constant.
  • Proposing a regex filter as the safe option — it is the least validated of the modes.
  • Putting a switch over the enum inside the test body, duplicating the production mapping.
  • Claiming exhaustive enum tests are always correct, ignoring the pressure they create toward vacuous assertions.
  • Assuming a renamed constant will be caught by any @EnumSource filter, regardless of mode.

context