skip to content

When would you write a custom JUnit 4 `Runner`, and what do you actually implement — what roles do `Description`, `RunNotifier` and `ParentRunner` play?

level: seniorimportance: nice to knowfreq 18%

answer

  1. Runner = getDescription() + run(RunNotifier)
  2. Description = immutable tree, read before execution
  3. RunNotifier = started/finished/failure/ignored events
  4. ParentRunner: getChildren / describeChild / runChild
  5. BlockJUnit4ClassRunner hooks: createTest(), methodBlock()

basics

~20 s

Runner is abstract with getDescription() (the tree of tests) and run(RunNotifier) (execute, firing started/finished/failure events). In practice you extend ParentRunner — implementing getChildren, describeChild, runChild — or subclass BlockJUnit4ClassRunner to tweak instance creation or validation.

solid answer

~50 s

You write a runner when the *shape* of the test class differs from "public void @Test methods on a freshly constructed instance" — tests defined by data or by an external file, a different instantiation model (instances supplied by a container), extra class-level validation, or aggregation of other classes. The contract is small: - `Description getDescription()` — an immutable tree (suite nodes and test nodes with display names and annotations) that tools read *before* execution to draw the test tree. - `void run(RunNotifier notifier)` — execute, calling `fireTestStarted`, `fireTestFinished`, `fireTestFailure(Failure)`, `fireTestIgnored`, `fireTestAssumptionFailed`. You rarely implement those directly. `ParentRunner<T>` already handles descriptions, filtering, sorting, `@BeforeClass`/`@AfterClass` and `@ClassRule`; you supply `getChildren()`, `describeChild(T)` and `runChild(T, RunNotifier)`. `BlockJUnit4ClassRunner extends ParentRunner<FrameworkMethod>` and is the right superclass when you only want to change instance creation (`createTest()`), the per-method statement (`methodBlock()`), or validation. Remember the constructor requirement: a public constructor taking `Class<?>`.

code

java · 28 lines
java
public class SpecRunner extends ParentRunner<File> {

    private final List<File> specs;

    public SpecRunner(Class<?> testClass) throws InitializationError {
        super(testClass);
        this.specs = Specs.discover(testClass);
    }

    @Override protected List<File> getChildren() { return specs; }

    @Override protected Description describeChild(File spec) {
        return Description.createTestDescription(getTestClass().getJavaClass(), spec.getName());
    }

    @Override
    protected void runChild(File spec, RunNotifier notifier) {
        Description description = describeChild(spec);
        notifier.fireTestStarted(description);
        try {
            Specs.execute(spec);
        } catch (Throwable t) {
            notifier.fireTestFailure(new Failure(description, t));
        } finally {
            notifier.fireTestFinished(description);
        }
    }
}

go deeper

for a junior

Know that a runner is what executes a test class and that custom ones are rare; naming getDescription/run is enough.

for a middle

Explain Description versus RunNotifier and that ParentRunner/BlockJUnit4ClassRunner are the practical superclasses.

for a senior

Choose the right superclass, implement the three ParentRunner methods correctly, handle notifier pairing and filtering, and know when a rule is the right tool instead.

for a principal

Weigh a custom runner as an API commitment — it consumes the class's only extension seam, must keep IDE integration honest, and is maintenance the team owns forever.

## The contract `org.junit.runner.Runner` is abstract with two methods: ```java public abstract Description getDescription(); public abstract void run(RunNotifier notifier); ``` That is the entire class-level extension point of JUnit 4. ### Description An immutable tree node describing a test or a group of tests: a display name, optionally the test class and method name, the annotations on that element, and children. You build them with `Description.createSuiteDescription(name)` and `Description.createTestDescription(clazz, methodName)`. Tools consume the tree *before* execution — that is how an IDE renders every test in a class before a single one runs, and how filtering works (a `Filter` decides per `Description`). Two descriptions are equal by display name, so duplicate names collapse into one node in reports — a real trap for generated tests. ### RunNotifier The event bus. As the runner executes it calls `fireTestStarted(Description)`, `fireTestFinished(Description)`, `fireTestFailure(new Failure(description, throwable))`, `fireTestIgnored(Description)` and `fireTestAssumptionFailed(...)`. Listeners — IDE panels, report writers — subscribe. Anything you fail to report simply does not exist for tooling: a test that runs but never fires `fireTestStarted` is invisible, and a swallowed exception silently becomes a pass. Pairing started/finished correctly, including on the exception path, is the most common bug in hand-written runners. ## ParentRunner — the template you should start from `ParentRunner<T>` implements `run` and `getDescription` for you and asks for three methods: ```java protected abstract List<T> getChildren(); protected abstract Description describeChild(T child); protected abstract void runChild(T child, RunNotifier notifier); ``` In exchange it provides: the class-level statement (`@BeforeClass`, `@AfterClass`, `@ClassRule`), `Filterable` and `Sortable` support so IDE "run single method" works, child ordering, and validation error collection into `InitializationError`. `T` is whatever a child is: `FrameworkMethod` for a method-based runner, `Runner` for a suite. The two runners in the box demonstrate both shapes: `Suite extends ParentRunner<Runner>`, and `BlockJUnit4ClassRunner extends ParentRunner<FrameworkMethod>`. ## BlockJUnit4ClassRunner — extend this for small changes Most "custom runners" only need to tweak the default behaviour, and the hooks are protected methods: - `createTest()` — return the test instance from somewhere else (a container, a factory) instead of the no-arg constructor. - `methodBlock(FrameworkMethod)` — wrap or replace the statement chain for a method. - `withBefores` / `withAfters` / `possiblyExpectingExceptions` / `withPotentialTimeout` — finer-grained pieces of that chain. - `collectInitializationErrors` / `validateTestMethods` — relax or add class validation, for example to allow test methods with parameters. - `computeTestMethods()` — change which methods count as tests. ## The constructor requirement JUnit instantiates the runner reflectively, so it needs a public constructor taking `Class<?>` — or `Class<?>` plus `RunnerBuilder` when it builds runners for other classes. Validation problems should be thrown as `InitializationError` carrying the list of causes; that is what produces the readable "initializationError" entry instead of a mysterious stack trace. ## When it is worth it - Tests defined by data outside Java (a directory of specification files, a matrix from a resource) where each file becomes a child. - A different instantiation model — instances built by a dependency-injection container. - Class-level policy: extra validation, custom ordering, environment-based skipping. - Aggregation shapes that `Suite` does not cover. ## When it is not If the requirement is "do something around each test", that is a **rule**, not a runner — rules compose and runners do not, and a class can hold only one runner. Writing a runner to add setup/teardown is the classic over-engineering mistake here, and it permanently blocks the class from using any other runner such as `Parameterized`. ## Practical cautions Hand-written runners are where IDE integration breaks: unimplemented filtering means "run one test method" runs the whole class; duplicated display names collapse in reports; and unmatched started/finished events leave tests stuck as "running". Extending `ParentRunner` rather than `Runner` avoids nearly all of it, because filtering, sorting and event pairing are already correct. ## Interview framing State the two-method contract, define `Description` and `RunNotifier` in one line each, then immediately say you would extend `ParentRunner` or `BlockJUnit4ClassRunner` rather than implement `Runner` directly — and note that a runner is the wrong tool when a rule would do.

  • Why extend `ParentRunner` instead of implementing `Runner` directly?
    `ParentRunner` already implements `run` and `getDescription` correctly and adds the class-level statement (`@BeforeClass`, `@AfterClass`, `@ClassRule`), child ordering, and `Filterable`/`Sortable` support. Filtering in particular is what lets an IDE run a single test method; a raw `Runner` that ignores it runs the whole class instead, which users perceive as a broken integration.
  • What goes wrong if a custom runner does not fire the notifier events correctly?
    Tooling only knows what the notifier is told. Skipping `fireTestStarted` makes the test invisible in reports; skipping `fireTestFinished` on the exception path leaves it displayed as still running; and catching a throwable without `fireTestFailure` turns a genuine failure into a silent pass. Always pair started/finished in a `try`/`finally` and report failures explicitly.

saying these in an interview costs you the question

  • Writing a runner to add setup and teardown around tests, where a rule composes and a runner permanently claims the class's only `@RunWith` slot.
  • Forgetting the public `Class<?>` constructor, producing an initialization error.
  • Implementing `Runner` directly and losing filtering, so "run a single test method" in the IDE runs everything.
  • Generating children with duplicate display names — descriptions are equal by name and collapse in reports.
  • Catching a test's exception without firing `fireTestFailure`, silently turning failures into passes.

context