skip to content

What does Cucumber's --publish option actually do, and what do you give up by turning it on?

level: middleimportance: nice to knowfreq 23%

answer

  1. an upload, not a formatter
  2. the same messages, sent elsewhere
  3. a link printed at the end
  4. quiet and disabled are different switches
  5. the token belongs in the environment

basics

~20 s

--publish uploads the run's message stream to Cucumber's hosted reports service and prints a link to the rendered report. In exchange, run data leaves your network, anyone holding the link can read it, and the run gains a network dependency.

solid answer

~40 s

Publishing is an upload, not a formatter. At the end of a run Cucumber sends the same messages a local formatter would consume to the hosted reports service, which renders them and returns a URL that is printed in the log. In Cucumber-JVM it is switched on by `--publish` or by setting `cucumber.publish.enabled` to true, and a token supplied through `cucumber.publish.token` or the `CUCUMBER_PUBLISH_TOKEN` environment variable ties the report to an account instead of publishing anonymously. `cucumber.publish.quiet` silences the banner that advertises the service, which is a **different switch from disabling the upload** and is the confusion most teams hit. What you give up is control: the data leaves your network, the link is unguessable rather than access-controlled, and a slow or unreachable service now touches your build.

code

properties · 2 lines
properties
cucumber.publish.enabled=false
cucumber.publish.quiet=true

go deeper

for a junior

Recall that publishing uploads the run to a hosted service and prints a link, and that it is off unless someone asks for it. Know the banner in the log is an advert, not a confirmation.

for a middle

Explain the two independent switches, one for the upload and one for the banner, and where the token belongs. Be able to say what content actually leaves the machine.

for a senior

Show the operational judgement: a network call at the end of every run, no retention you control, and a link that grants read access to whoever holds it. Nightly reports belong in archived artefacts.

for a principal

Own it as a data-egress policy decision rather than a convenience toggle, written once into shared configuration so every new module inherits it without another argument.

The publish feature is small, easy to enable by accident, and the one part of Cucumber reporting with a compliance conversation attached to it. ## What publishing actually moves Publishing is not another output format. The run finishes, and Cucumber sends the same messages a local formatter would have consumed to a hosted service, which renders them into a browsable report and hands back a URL that is printed to the log. The content is therefore everything the stream carries: feature file text, scenario and step names, tags, statuses, durations, error messages and any attachments the run made. That list is worth reading twice before enabling it. Scenario names in a climbing-gym membership product are usually harmless. Error messages that quote a failing request body are not always harmless, and neither is an attachment. ## The switches | Concern | Cucumber-JVM | cucumber-js | |---|---|---| | turn it on for a run | `--publish` on the command line | `--publish` on the command line | | turn it on in configuration | `cucumber.publish.enabled=true` | the equivalent key in its config file | | identify an account | `cucumber.publish.token`, or `CUCUMBER_PUBLISH_TOKEN` in the environment | `CUCUMBER_PUBLISH_TOKEN` in the environment | | silence the advertising banner | `cucumber.publish.quiet=true` | its publish-quiet configuration key | Two rules follow from that table: 1. **The token belongs in the environment, never in a committed properties file.** It is a credential that lets anything holding it publish under your account. 2. **Quiet is not off.** Silencing the banner stops a message; it does not stop an upload, and disabling the upload is a separate setting. A team that only sets the quiet flag and believes it has opted out has not. ## What you give up - **Locality.** Run data crosses your network boundary to a third party. In a regulated environment that is a decision someone other than the test author gets to make. - **Access control.** An anonymous report link is unguessable but not gated: anyone who has it can read the whole report. A token ties reports to an account, where access follows the account rather than link-holding. - **Retention.** How long an anonymously published report stays available is the service's policy, not yours. Nothing about it is an archive you control. - **Build independence.** A publish step is a network call at the end of every run. When it is slow or unreachable, the run's tail becomes flaky in a way that has nothing to do with your product. - **A single source of truth.** If the published report is the one people open and the local artefacts are never archived, you have outsourced your evidence. ## When it earns its place It is genuinely good at exactly one thing: getting a rich, readable report in front of someone who cannot open your CI system, immediately and with no infrastructure. A contractor debugging a failing branch, a pair looking at one run together, a demonstration. For those the alternative is screenshots of a console, and publishing wins. It is a poor fit as the standing report for a nightly run. A nightly run's report needs to be archived, comparable across runs, and reachable by policy rather than by whoever kept the link. That is a CI artefact plus a formatter, not an upload. Take the climbing-gym membership product's 26-file feature directory. A contractor debugging one membership-renewal failure on a branch, with no access to your CI system, is a genuine case: one run with publishing on, a link pasted into the ticket, a minute's work and no infrastructure. The nightly run at 02:15 is the opposite case. Its report has to be archived, comparable with last week's, and reachable by anyone on the team by policy rather than by whoever still has the link in a chat message. That is a formatter plus a build artefact, not an upload. ## The banner is not the feature Most engineers meet publishing through the banner Cucumber prints inviting them to share a report, and conclude either that reports are already being uploaded, or that the tool is nagging them. Neither reading is useful. Decide the policy explicitly for the repository: publishing on with a token for a defined purpose, or publishing off and the banner quiet, written into the checked-in configuration so a new module inherits the decision instead of re-litigating it.

  • A team set the publish-quiet setting and told an auditor that publishing is disabled. Are they right?
    No. The quiet setting suppresses the message that advertises the service; whether anything is uploaded is decided by the enabled setting or by passing the publish option. They are two independent switches, and only the second one is an opt-out. The auditable answer is an explicit enabled value in checked-in configuration.
  • What changes about a published report when a token is supplied?
    An anonymous publish produces a standalone link that anyone holding it can open, retained under the service's default policy. A token attaches the report to an account, so reports are grouped together, access follows account membership rather than link-holding, and retention follows that account's terms.

saying these in an interview costs you the question

  • Thinks the publish banner means reports were already uploaded
  • Puts the publish token in a committed properties file
  • Assumes a published report link is access-controlled
  • Treats publishing as a substitute for archiving CI artefacts
  • Cannot distinguish silencing the banner from disabling the upload