skip to content

How do you wire Allure or Serenity BDD onto a Cucumber-JVM run, and what must your CI job still do itself?

level: seniorimportance: should knowfreq 51%

answer

  1. one attaches, the other wraps
  2. results files are not the report
  3. something must render them afterwards
  4. clean the directory between runs
  5. generate even when the tests failed

basics

~20 s

Allure attaches as an ordinary Cucumber plugin, writing intermediate result files that the Allure command line later renders into HTML. Serenity BDD wraps the run and needs a separate aggregation step. CI must clear stale results and archive the output.

solid answer

~50 s

The two integrate at different layers. **Allure** ships a Cucumber plugin class that implements Cucumber's own plugin interface, so it is registered exactly like a built-in formatter through `--plugin` or the `cucumber.plugin` property. It subscribes to the same run events and writes one intermediate result file per test into a results directory; those files are **not a report**, and the Allure command line renders them afterwards. **Serenity BDD** is not a formatter: it wraps the runner, reads `serenity.properties`, writes its own per-test output, and the build's aggregate goal turns that into an HTML site after the test phase. Either way three things stay yours: clear the results directory between runs, run the generation step **even when tests failed**, and archive the output as a build artefact, because neither tool ships the report anywhere by itself.

code

java · 17 lines
java
package gym.report;

import io.cucumber.plugin.ConcurrentEventListener;
import io.cucumber.plugin.event.EventPublisher;
import io.cucumber.plugin.event.TestCaseFinished;

public class SlowScenarioListener implements ConcurrentEventListener {

    @Override
    public void setEventPublisher(EventPublisher publisher) {
        publisher.registerHandlerFor(TestCaseFinished.class, this::onFinished);
    }

    private void onFinished(TestCaseFinished event) {
        System.out.println(event.getTestCase().getName() + " -> " + event.getResult().getStatus());
    }
}

go deeper

for a junior

Recall that a third-party reporter is added like any other Cucumber plugin and that it writes intermediate files, so a second command turns those into the HTML you actually open.

for a middle

Explain the difference between a reporter that attaches as a plugin and one that wraps the runner, and name the steps the build owns: cleaning results, generating, archiving.

for a senior

Demonstrate the operational failures: stale results merged into today's report, generation skipped by a failing test phase, a reporter that is not safe under parallel execution.

for a principal

Own the choice itself. Adopting a reporting product is a long-lived dependency on your run's shape, so keep the raw message stream archived alongside it and be able to say who reads the report and why.

Every team eventually replaces the built-in HTML report with something that has history, trends and richer detail. The wiring question separates people who have run one in anger from people who have copied a snippet. ## Two different integration shapes | | Allure | Serenity BDD | |---|---|---| | Where it attaches | as a Cucumber plugin, like a built-in formatter | around the runner, replacing how the run is driven | | How it is registered | `--plugin` or the `cucumber.plugin` property | build dependencies plus `serenity.properties` | | What it consumes | Cucumber's run events, as they happen | the run it wraps, plus its own step instrumentation | | What it emits during the run | one intermediate result file per test, in a results directory | its own per-test output files | | What builds the HTML | a separate render step from the Allure command line | the build plugin's aggregate goal, after the test phase | The practical consequence of that first row is what interviewers listen for. Allure can be added to an existing run by appending one plugin entry and changing nothing else. Serenity is a bigger commitment: it wants to own the run, which is why it can report on things a plugin cannot see, and why it is harder to remove. ## Registering a third-party formatter A third-party reporter is not special. Cucumber exposes a plugin interface, `io.cucumber.plugin.EventListener`, whose implementation is handed an `EventPublisher` and registers handlers for the events it cares about. Anything registered through `--plugin` gets the same events the built-in `html` and `message` formatters get. Once you have seen how small that interface is, the reporting ecosystem stops looking like magic: Allure's Cucumber integration is a listener that turns events into files on disk. That also means you can write your own listener in an afternoon when the requirement is narrow, for example printing the slowest scenarios of a nightly run rather than adopting a whole reporting product. ## Why the concurrent variant exists Cucumber also exposes `io.cucumber.plugin.ConcurrentEventListener`. A plain `EventListener` is fed events that Cucumber has serialised into a stable order, which costs buffering; a `ConcurrentEventListener` receives them as they occur, from several threads, and must be thread-safe itself. A reporter that writes one file per scenario under scenario-level parallel execution wants the concurrent form, and a reporter that accumulates run state in a static field will interleave two scenarios' data and produce a confidently wrong report. When you evaluate a third-party reporter, whether it supports parallel execution is a real question, not a formality. ## What stays your job 1. **Clear the results directory before each run.** The generator renders whatever result files it finds. This is why a nightly report can show scenarios that were deleted a fortnight ago: nothing ever removed their result files from the workspace. 2. **Generate the report even when tests fail.** The whole point is reading it after a failure, so the render step must run in the build phase that executes regardless of test outcome, not one skipped by a failing test phase. 3. **Archive the output.** Neither tool publishes anything. If the build does not keep the directory as an artefact or copy it somewhere durable, the report lives exactly as long as the agent's workspace. 4. **Keep the raw message stream too.** A third-party reporter is a rendering choice. Archiving Cucumber's own NDJSON alongside it means a bad reporting upgrade is recoverable and a new question can be answered against old runs. 5. **Keep the integration artefact matched to your Cucumber major version.** These integrations bind to Cucumber's plugin API, and a mismatch fails at run start rather than degrading gracefully. ## A worked case A climbing-gym membership product has a 26-file feature directory and a nightly run at 02:15 that nobody reads any more. Adding Allure will not fix that, and the reason is instructive: the report was ignored because 41 failures were the same three environment problems, not because the HTML lacked charts. Wire the reporter for the audience you can name and the question you can state, and fix the run first. A richer report over an untrusted run just makes the noise prettier.

  • An Allure report from the nightly run lists scenarios that were deleted a fortnight ago. Why?
    The results directory was never cleared, so result files from earlier runs are still on the agent and the generator renders everything it finds. Clean the directory as the first step of the run, or use a fresh workspace per build. Nothing about the deletion of a feature file removes yesterday's result files.
  • Why does a reporter used under scenario-level parallel execution need the concurrent listener interface?
    It is handed events as they happen from several threads rather than pre-serialised into one order, so it must be thread-safe and must key its bookkeeping by test case rather than holding run state in fields. A reporter written for serial execution will interleave two scenarios and report confidently wrong results.
  • You add Serenity BDD and the HTML site is empty even though tests ran. What do you check first?
    Whether the aggregation step ran at all. Serenity produces per-test output during the run and builds the site in a separate step afterwards, and that step is commonly bound to a phase the build skipped, or skipped itself because the test phase failed. Check the build log for the aggregate goal before suspecting configuration.

The reporter is a stenographer in the room rather than a historian writing afterwards: it hears the run as it happens and leaves notes, and something else still has to bind those notes into a book.

saying these in an interview costs you the question

  • Thinks Allure produces the finished HTML during the test run
  • Never clears the results directory between nightly runs
  • Generates the report only when the test phase succeeded
  • Assumes Serenity attaches through the plugin option like a formatter
  • Keeps a reporter's run state in a static field under parallel runs
  • Never archives the report directory as a build artefact