skip to content

Build Plugin Wrappers

How a build starts a run instead of a person - a third-party Maven or Gradle wrapper, a community container image, or a plain jmeter -n step - and why Apache publishes none of them.

on this pageshow

explore

questions

5

A nightly job runs jmeter -n -t plan.jmx -l results.jtl -e -o report and it fails on the second run. Why?

level: middleimportance: must knowfreq 58%

answer

  1. Second run, first run's leftovers
  2. One artefact is checked, one is opened
  3. Ask when the report folder is validated
  4. There is a flag whose name says delete

basics

~20 s

Last night's report directory is still there and is not empty, so JMeter refuses to write into it and aborts before the plan is even loaded. Adding -f deletes the old report folder and results file first.

solid answer

~40 s

JMeter validates the `-o` folder while it is still parsing the command line, before `startNonGui` loads the plan. `JOrphanUtils.canSafelyWriteToFolder` throws `IllegalArgumentException: Cannot write to '<path>' as folder is not empty` the moment it finds any entry there, so the second night's run dies without producing a single sample. The results file fails in the opposite, quieter way: `ResultCollector` opens the JTL with `new FileOutputStream(filename, append)` where `append` is simply whether the file already exists, so a CSV JTL is appended to and its header is not rewritten. `-f` (`--forceDeleteResultFile`) fixes both — the folder check then deletes the directory recursively and recreates it, and any existing results file is deleted before the engines start. Giving each run its own directory works too.

code

bash · 13 lines
bash
# night one: clean workspace, both artefacts created
jmeter -n -t plan.jmx -l results.jtl -e -o report

# night two, same workspace:
#   An error occurred: Cannot write to '/build/ws/report' as folder is not empty
#   (thrown while parsing options - plan.jmx is never loaded)

# fix A: reset both artefacts up front
jmeter -n -f -t plan.jmx -l results.jtl -e -o report

# fix B: never reuse a path
run="out/${BUILD_NUMBER}"
jmeter -n -t plan.jmx -l "${run}/results.jtl" -e -o "${run}/report"

go deeper

for a junior

Recall that a reused workspace is the usual reason a second nightly run fails, and that -f exists to clear the previous report folder and results file before the test starts.

for a middle

Explain the asymmetry: the report folder is validated up front and refuses a non-empty directory, while the results file is simply opened in append mode. Name the flag and say what each half of it deletes.

for a senior

Show you would catch the quiet half. Appended JTLs make a report describe several runs as one, so state how you would notice it and whether you prefer -f or a per-build output path.

for a principal

Decide the estate's convention: reset in place or write to an immutable per-run path, and who owns workspace hygiene on shared agents. The trade is disk and retention against reproducibility of an archived artefact.

## Two artefacts, two different failure modes The command in the question writes two things a rerun can collide with: the sample log named by `-l`, and the dashboard folder named by `-o`. They behave nothing alike. | Artefact | Already exists, no `-f` | Already exists, with `-f` | |---|---|---| | `-o report` folder, non-empty | `IllegalArgumentException: Cannot write to '<path>' as folder is not empty`; run aborts | Deleted recursively, then recreated | | `-o report` exists as a *file* | `... as it is an existing file`; run aborts | The file is deleted | | `-l results.jtl` | Opened in append mode, no header rewritten | Deleted before the engines start | | `jmeter.log` | Truncated and rewritten every run | Same; `-f` is irrelevant to it | So the loud failure and the silent one live in the same command line, and the loud one is what you actually see on night two. ## Where the folder check happens This matters because it determines what you can conclude from the logs. In JMeter's CLI branch, `extractAndSetReportOutputFolder(parser, deleteResultFile)` runs **before** `startNonGui(...)`. It resolves `-o` to an absolute path, calls `JOrphanUtils.canSafelyWriteToFolder`, and only then sets the property `jmeter.reportgenerator.outputdir` that both exporters read. The consequences worth remembering: - The plan is never parsed, so a JMX error and a dirty workspace look nothing alike in the log. - No JTL is created, so there is no partial results file to inspect. - The check is a plain existence-and-emptiness test. A stray `.gitkeep` or a leftover editor swap file in `report/` is enough to stop the run. - The exporter re-checks the folder later with a *narrower* filter that only looks for `index.html`, `content` and a `sbadmin2-*` directory. That second check is more permissive than the first, so the command-line check is always the one that bites. ## Why the JTL is the more dangerous half A failed build is annoying; a green build reporting the wrong thing is worse. `ResultCollector.getFileWriter` computes, for a CSV log, `trimmed = new File(filename).exists()` and then opens `new FileOutputStream(filename, trimmed)`. The second argument is the append flag. It also writes the CSV header only when `trimmed` is false. Put plainly: 1. Night one creates `results.jtl` with a header and its rows. 2. Night two appends its rows to the same file, with no second header and no separator. 3. Anything that later reads that file — including the dashboard generator — treats the rows of both nights as a single result set. For an XML log the mechanics differ (the closing `</testResults>` terminator is trimmed off the existing file and the new samples are appended in its place, the terminator being rewritten when the file is closed) but the outcome is the same: accumulation, not replacement. ## Fixing it Two defensible fixes, and the choice is about what the job archives: - **Add `-f`.** One flag, and both artefacts are reset. `-f` sets an internal flag that makes the folder check delete the directory tree and recreate it, and that makes the run delete every `ResultCollector` file before the engines start. If a delete fails, you get `IllegalStateException: Could not delete results file ...`, which is again a hard stop rather than a silent carry-over. - **Write into a fresh directory per run.** Point `-l` and `-o` at a path containing the build number or a timestamp. Nothing is ever reused, the collision cannot occur, and the archive step has a naturally unique name to upload. The cost is that the agent's disk grows unless the workspace is cleaned or the paths are inside a directory the job wipes. A nightly job that archives its report usually wants both properties: a clean start *and* a distinguishable output path. The habit worth building is to treat the workspace as dirty by default, because on a reused agent it is. One last trap: `-e` is refused outright without `-l`, with `Option -e requires -l option`. A build step that stops writing the JTL but keeps `-e` will not quietly skip the report — it will complain about usage.

  • The job now passes with -f, but the archived report covers a longer run than the plan schedules. What happened before -f was added?
    The JTL was appended to on every rerun, because JMeter opens an existing sample log in append mode and skips the header. The dashboard was generated from that accumulated file, so it aggregated several nights as one run. `-f` deletes the results file before the engines start, which is what makes the report describe a single run again.
  • Does -f help with jmeter.log as well?
    No, and it does not need to. The shipped `log4j2.xml` declares its file appender with `append="false"` against `jmeter.log`, so each run truncates and rewrites it. If a nightly job wants to keep last night's run log it must archive the file or give each run its own name with `-j`, not rely on `-f`.
  • Why did the failure message name no line of the test plan?
    Because the plan had not been read yet. The `-o` folder check runs during command-line processing, before `startNonGui` opens the JMX. A dirty workspace therefore produces a message about a directory and nothing about the plan, which is a useful way to tell the two classes of failure apart.

saying these in an interview costs you the question

  • Says JMeter overwrites an existing results file
  • Blames the JMX file for a dirty-workspace failure
  • Thinks the run aborts only after sampling has started
  • Expects -f to also reset jmeter.log
  • Assumes an empty-looking folder with a dotfile passes the check
open as a page

Which Maven or Gradle plugin does the Apache JMeter project itself publish for running a plan in a build?

level: juniorimportance: should knowfreq 42%

basics

~10 s

None. Apache publishes JMeter only as a binary distribution, apache-jmeter-6.0.0.tgz or .zip, plus library jars under the org.apache.jmeter group. Every Maven, Gradle, Jenkins and container wrapper around it is a third-party community project.

open as a page

In a build step, why does declaring a JMeter plugin jar as a build dependency not make its element available?

level: middleimportance: should knowfreq 40%

basics

~10 s

JMeter is launched with java -jar ApacheJMeter.jar, and java ignores CLASSPATH and -cp whenever -jar is used. Components are found only under the installation's lib/ext directory, or wherever the search_paths property points.

open as a page

What should a nightly JMeter job archive from a run so its results stay usable weeks later?

level: seniorimportance: should knowfreq 46%

basics

~20 s

Archive the whole folder named by -o, not just index.html, along with the JTL results file and the run's jmeter.log. The dashboard is a directory tree of index.html, content, an sbadmin2 theme folder and statistics.json.

open as a page

Across a dozen pipelines, would you standardise JMeter runs on a third-party build wrapper or a plain jmeter -n step?

level: principalimportance: should knowfreq 34%

basics

~20 s

Either can work, so decide it on ownership. A community wrapper hides provisioning but puts someone else's release cadence and JMeter version pin between you and the tool; a plain step makes provisioning explicit, longer, and yours to maintain.

open as a page