skip to content

A suite launched with `Runner.path("classpath:api").parallel(4)` writes an HTML report, but the CI job reports "no tests found" because it can find no XML. Which Karate builder defaults explain that, and where does Karate write its output?

level: middleimportance: should knowfreq 52%

answer

  1. One format is on, two are off
  2. Karate writes to its own directory
  3. Not where the surefire plugin writes
  4. The old directory is renamed, not deleted
  5. One XML file per feature

basics

~20 s

Karate's HTML report is on by default but JUnit XML and Cucumber JSON are off; you must call outputJunitXml(true). Output goes to karate-reports under the build directory, not to the surefire directory CI usually reads.

solid answer

~30 s

Two defaults collide here. First, on the Karate builder `outputHtmlReport` defaults to **true** while `outputJunitXml` and `outputCucumberJson` default to **false**, so a chain that never mentions them produces HTML and nothing machine-readable — `.outputJunitXml(true)` is the fix. Second, Karate writes to its **own** directory: `karate-reports` under the build directory, which is `target/karate-reports` under Maven and `build/karate-reports` under Gradle. A CI step configured to collect `target/surefire-reports/*.xml` will find only the one file Surefire wrote for the launcher class itself, not the per-feature XML Karate produced. Point the collector at `**/karate-reports/*.xml`, or move the directory with the builder's output-directory call.

code

java · 7 lines
java
Results results = Runner.path("classpath:api")
        .outputJunitXml(true)      // off by default - CI needs this
        .outputCucumberJson(true)  // also off by default
        .backupReportDir(false)    // v2 name: backupOutputDir
        .reportDir("target/karate-reports") // v2 name: outputDir
        .parallel(4);
assertEquals(0, results.getFailCount(), results.getErrorMessages());

go deeper

for a junior

Recall that HTML comes for free but JUnit XML does not, and that Karate writes into karate-reports under the build directory rather than alongside the surefire output.

for a middle

Explain each switch and its default, and that the output directory is renamed with a timestamp on the next run rather than cleared, so stale results can survive.

for a senior

Trace the whole path from suite to dashboard: the format switch, the collector glob, the backup directories and the single surefire result that can mask a red suite.

for a principal

Decide where the reporting contract lives — a runner class that hard-codes formats and directories binds every pipeline to one shape, and moving it later means touching every repository.

## The three output switches, and their defaults The Karate `Runner.Builder` exposes each report format as its own boolean, and they do **not** default the same way: | Builder call | Default | Produces | |---|---|---| | `outputHtmlReport(boolean)` | **true** | the human-facing report, entry point `karate-summary.html` | | `outputJunitXml(boolean)` | **false** | one `<testsuite>` XML file per feature | | `outputCucumberJson(boolean)` | **false** | one Cucumber-shaped JSON file per feature | This is the whole explanation for the symptom in the question. The suite really did run, really did pass or fail, and really did write a report — just not one the CI job knows how to read. Adding `.outputJunitXml(true)` to the chain makes the XML appear. The XML files are named after each feature's package-qualified path, so a feature at `api/orders.feature` becomes something like `api.orders.xml`. **There is one file per feature, not one per suite** — collectors that expect a single result document need a glob, not a filename. ## Where the files land The default output directory is `karate-reports`, resolved under the build directory: - **`target/karate-reports`** under Maven - **`build/karate-reports`** under Gradle Karate works out which by inspecting the build environment, so you normally get the right one without configuring anything. To move it, use the builder's output-directory call — and note that this is one of the names that changed between Karate's two lines: | | Karate 1.x | Karate 2.x | |---|---|---| | set the directory | `reportDir(String)` | `outputDir(String)` | | keep or discard the previous run | `backupReportDir(boolean)` | `backupOutputDir(boolean)` | Karate 2 keeps a compatibility shim so a 1.x chain calling `reportDir(...)` still compiles, but new code on 2.x should say `outputDir(...)`. ## The backup default, which surprises people Backing up the report directory defaults to **true** in both lines. Before a run starts, if the output directory already exists it is **renamed** with a timestamp suffix rather than deleted or overwritten. After three local runs you have: ``` target/karate-reports target/karate-reports_1757462391044 target/karate-reports_1757462455310 ``` Two consequences worth knowing: 1. **Nothing is ever cleaned up.** On a long-lived build agent that does not wipe its workspace, these accumulate run after run. 2. **A greedy collector picks up stale results.** A CI glob of `target/karate-reports*/*.xml` will happily hoover up every previous run's XML alongside the current one, producing a test count that grows every build. Match the directory exactly, or turn the backup off in CI with `backupReportDir(false)` on 1.x / `backupOutputDir(false)` on 2.x so each run starts clean. ## Wiring a CI job correctly The practical checklist, in the order things usually go wrong: 1. **Turn the machine-readable format on** — `.outputJunitXml(true)` in the runner chain. Without it there is nothing to collect no matter where you look. 2. **Point the collector at Karate's directory**, not Surefire's. `**/karate-reports/*.xml` rather than `target/surefire-reports/*.xml`. 3. **Decide about the backup directories.** Either exclude them from the glob or disable backups on the agent. 4. **Keep the HTML too.** It is on by default and it is the artefact a human actually opens when the XML says a scenario failed; archive the directory rather than only the XML files. 5. **Remember the launcher is also a JUnit test.** Surefire writes its own result for the runner class, which is why a job can report "1 test, 0 failures" while the Karate suite ran two hundred scenarios — the assertion inside the launcher collapsed all of them into one JUnit result. That last point is the one that hides real failures. If the launcher never asserts on the returned failure count, Surefire's single green result is all CI ever sees, and the red scenarios sit quietly in an HTML report nobody opened. ## What the switches do not control The builder decides *which* formats are emitted and *where*; it does not shape their content. The XML is a plain JUnit `<testsuite>` document with one `<testcase>` per scenario, and the JSON is the Cucumber-compatible shape — both exist so that existing tooling can read a Karate run without knowing anything about Karate. If a downstream tool needs richer structure, it consumes those files; there is no builder switch that changes what goes into them. ## What else is in that directory The output directory is not just XML. A single run leaves behind: - **`karate-summary.html`** — the entry point of the HTML report, with a per-tag filter view. - **one HTML page per feature**, linked from the summary, showing every step and its payload. - **one XML file per feature** when JUnit XML is switched on, named after the feature's package-qualified path. - **one JSON file per feature** when Cucumber JSON is switched on. - **embedded assets** such as screenshots, referenced from the HTML pages rather than inlined. That matters for archiving: the HTML report is a **directory**, not a file, and archiving only `karate-summary.html` gives you a summary whose links all break. Archive the whole directory, and remember that a scenario tagged `@report=false` deliberately contributes no step detail to any of these outputs — it ran, but its steps are suppressed so a token or credential does not reach a shared report.

  • Why does a CI job show "1 test, 0 failures" when the Karate suite ran 200 scenarios?
    Because it is reading Surefire's result for the launcher class, which is a single JUnit test. Karate's per-scenario detail lives in its own XML under `karate-reports`, and only if `outputJunitXml(true)` was set. Collect that directory as well, and make sure the launcher asserts on the returned failure count so the single Surefire result at least turns red.
  • What happens to the previous run's report directory?
    It is renamed with a timestamp suffix rather than deleted, because backing up the report directory defaults to true. On a build agent that reuses its workspace these accumulate indefinitely, and a loose CI glob will collect stale XML from them. Disable it for CI with `backupReportDir(false)` on 1.x or `backupOutputDir(false)` on 2.x.

saying these in an interview costs you the question

  • Assuming Karate writes JUnit XML without being asked
  • Looking for Karate results in the surefire reports directory
  • Expecting a single XML file rather than one per feature
  • Thinking the previous report directory is overwritten or deleted
  • Believing the HTML report also needs to be switched on
  • Trusting a green surefire result for the launcher class