skip to content

Reporting & Living Documentation

2 roadmaps5 questions

How Cucumber's formatters emit HTML, JSON and the Messages stream, and how an Allure or Serenity adaptor is wired onto a run. Interviewers probe it because reporting is where BDD faces the business.

on this pageshow

explore

questions

5

In Cucumber, how do you select a formatter and point it at an output file, and which built-in formats are machine-facing?

level: juniorimportance: must knowfreq 66%

answer

  1. name first, destination after
  2. a colon separates format from path
  3. repeat the option for several formats
  4. no path means the console
  5. JVM says plugin, js says format

basics

~20 s

Cucumber-JVM registers a formatter with --plugin name:path or the cucumber.plugin property; cucumber-js uses --format name:path; Behave pairs --format with --outfile. Console and HTML output is written for people, while JSON, JUnit XML and message output is written for tools.

solid answer

~40 s

A formatter is chosen by name and given an output path after a colon. In **Cucumber-JVM** that is `--plugin html:target/cucumber-report.html`, or the `cucumber.plugin` property, which takes a comma-separated list. In **cucumber-js** it is `--format html:cucumber-report.html`, or the `format` entry in its config file. **Behave** splits it: `--format` picks the formatter and `--outfile` gives the destination, paired in order. Omit the path and the formatter writes to the console; repeat the option to run several at once, but only one may own standard output. The human-facing built-ins are `pretty` and `progress` on the console and `html` afterwards. The machine-facing ones are `message` (the canonical NDJSON stream), `junit` (JUnit XML for CI panels) and `json` (a legacy shape older report generators read).

code

properties · 2 lines
properties
cucumber.plugin=progress, html:target/cucumber-report.html, message:target/cucumber-messages.ndjson, junit:target/cucumber-junit.xml
cucumber.publish.enabled=false

go deeper

for a junior

Recall the shape name-colon-path and that omitting the path prints to the console. Know that no report file appears unless a formatter asks for one, and that the option is repeated per format.

for a middle

Explain why one human format and one machine format is the normal pairing, and what JUnit XML loses. Know the option name differs between Cucumber-JVM, cucumber-js and Behave.

for a senior

Show that a report only exists if the build archives it, and that the message stream is the artefact worth keeping because every other format can be rebuilt from it.

for a principal

Own the standard: one agreed formatter set across teams, one archived artefact, one place reports land. Divergent per-team formatter lists are why nobody can compare two suites.

A Cucumber run produces nothing but console noise unless you ask for a formatter, so this is the first configuration every automation engineer writes and the one interviewers check first. ## Selecting a formatter, per implementation | Implementation | Option | Shape | Config file equivalent | |---|---|---|---| | Cucumber-JVM | `--plugin` | `name:path` | `cucumber.plugin` in `cucumber.properties`, comma-separated | | cucumber-js | `--format` (`-f`) | `name:path` | the `format` list in its config file | | Behave | `--format` (`-f`) plus `--outfile` (`-o`) | two options paired in order | `behave.ini` | The spelling difference matters in an interview and in a terminal: `--plugin` is Cucumber-JVM's, `--format` is cucumber-js's and Behave's, and copying one into the other simply fails to start. Behave is the odd one out because the destination is a separate option, so `-f json -o report.json -f pretty` means the JSON goes to the file and the pretty output goes to the console. ## Where the output goes - **No path means the console.** `--plugin pretty` prints; `--plugin html:target/cucumber-report.html` writes a file and prints nothing. - **Repeat the option for several formatters.** One human format plus one machine format is the normal production pairing. - **Only one formatter may own standard output.** cucumber-js fails the run outright if two formatters are left without a path, rather than interleaving them into unusable text. - **Paths are resolved from the working directory**, which the build tool decides. That is why the same value lands in `target/` under one build and `build/` under another. ## Human-facing versus machine-facing built-ins | Format | Audience | What it produces | |---|---|---| | `pretty` | people, live | the Gherkin steps as they run, with results inline | | `progress` | people, live | one character per step, cheap in a CI log | | `html` | people, after the run | one self-contained page embedding the run's messages | | `message` | machines | the raw NDJSON event stream, the canonical machine format | | `junit` | machines | JUnit XML that CI test-result panels understand | | `json` | machines | a legacy structured shape older third-party generators read | | `rerun` | machines | a list of failed scenario locations, fed back to re-run only those | | `usage` | people | step definitions with hit counts and timings, useful for finding dead glue | The human-facing ones are ephemeral by nature. `pretty` output lives in a CI log that rotates; `html` is a file that must be archived by the build to survive. The machine-facing ones are inputs to something else: `junit` exists so a CI server can colour a test tab, not so a person can read behaviour, and it flattens each scenario into a single test case, losing the step-by-step structure that made the report worth reading. ## A worked configuration A climbing-gym membership product runs a 26-file feature directory nightly. A sound default is four entries: 1. `progress` on the console, so a 9-minute run does not produce 4,000 lines of log. 2. `html:target/cucumber-report.html` for anyone who opens the build. 3. `message:target/cucumber-messages.ndjson` as the archived artefact of record. 4. `junit:target/cucumber-junit.xml` so the CI test tab lists failures. Then the build archives `target/` as an artefact. Without that last step the HTML exists for the lifetime of the agent's workspace and no longer. ## The same list, wherever the run is driven from The formatter list is not really a property of the command line; it is a configuration value under one key. A command-line run passes it as repeated options, a properties file sets `cucumber.plugin` once for every run in the module, and a JUnit Platform suite supplies the same `cucumber.plugin` key as a configuration parameter. Deciding *where* that value lives is the choice that matters: a formatter list typed on one engineer's command line is a personal preference that dies with the terminal, while the same list in checked-in configuration is the team's standard, inherited by every new module without another discussion. ## Mistakes that cost a build - Expecting an HTML report without configuring one. Nothing is written by default. - Giving two formatters no path and wondering why the run refuses to start. - Treating JUnit XML as the stakeholder-facing report. It is a CI integration format; the scenario text does not survive it intact. - Writing the output somewhere the build never archives, so the report dies with the workspace. - Assuming the same option name works across implementations.

  • You configured two formatters and one of them printed nothing. What is the likely cause?
    Both were left without a destination, so both tried to own standard output. cucumber-js refuses to start in that case rather than interleaving them. The fix is to give every formatter after the first an explicit file path and keep exactly one on the console.
  • Why is JUnit XML a lossy way to report a Cucumber run?
    It exists to fill a CI test-result panel, so it flattens each scenario into one test case with a name and a pass or fail. The Gherkin step structure, per-step statuses, data tables and attachments do not survive the projection. Use it for the CI tab and keep a richer format for reading.
  • How do you re-run only the scenarios that failed last night?
    Configure the `rerun` formatter with a file path. It writes the locations of failed scenarios, and that file can be handed back to the next run as the set of things to execute. It is a machine-facing format whose consumer is the next invocation of Cucumber.

saying these in an interview costs you the question

  • Expects an HTML report without configuring any formatter
  • Gives two formatters no path and expects both on the console
  • Offers JUnit XML as the stakeholder-facing report
  • Uses --plugin with cucumber-js or --format with Cucumber-JVM
  • Writes reports to a path the build never archives
open as a page

What does a single line of Cucumber's Messages NDJSON stream represent, and what consumes that stream?

level: middleimportance: should knowfreq 37%

basics

~20 s

Each line is one JSON envelope holding exactly one message: a parsed feature file, a compiled runnable scenario, a step result, an attachment. Cucumber's built-in HTML, JSON and JUnit formatters all consume that same stream rather than writing independently.

open as a page

How do you wire Allure or Serenity BDD onto a Cucumber-JVM run, and what must your CI job still do itself?

level: seniorimportance: should knowfreq 51%

basics

~20 s

Allure attaches as an ordinary Cucumber plugin, writing intermediate result files that the Allure command line later renders into HTML. Serenity BDD wraps the run and needs a separate aggregation step. CI must clear stale results and archive the output.

open as a page

Several teams read one Cucumber suite's report - how do you decide between a bundled formatter and building on the Messages stream?

level: principalimportance: should knowfreq 28%

basics

~20 s

Start from the audience and the decision the report drives. Bundled formatters and tag-filtered runs cover most multi-team needs for free. Build on the message stream only when history, merging or per-team slicing survives those cheaper options.

open as a page

What does Cucumber's --publish option actually do, and what do you give up by turning it on?

level: middleimportance: nice to knowfreq 23%

basics

~20 s

--publish uploads the run's message stream to Cucumber's hosted reports service and prints a link to the rendered report. In exchange, run data leaves your network, anyone holding the link can read it, and the run gains a network dependency.

open as a page