skip to content

When a Gatling simulation's assertions fail, what exit code does the Gatling process return, and what are the other codes in that set?

level: juniorimportance: must knowfreq 62%

answer

  1. One integer is the whole verdict
  2. Three declared values, nothing else
  3. Zero success, one arguments, two assertions
  4. Assertion failure is exit two
  5. k6's ninety-nine belongs to k6

basics

~20 s

A failed assertion exits 2. Gatling declares exactly three status codes: 0 for success, 1 for invalid command-line arguments, and 2 for assertions failed. Other load generators differ; k6 exits 99 on a breached threshold.

solid answer

~40 s

Gatling's `StatusCode` type declares three values and only three: `Success` is `0`, `InvalidArguments` is `1`, and `AssertionsFailed` is `2`. A run whose `setUp` assertions all held exits `0`; a run where at least one did not hold exits `2`. `1` is specifically the command-line parser rejecting its arguments, not a general failure code. All assertions are evaluated after the simulation has finished, from the run's own recorded data, so the exit code is a single end-of-run verdict rather than something computed in flight. Two things it deliberately does not encode: how many requests failed, and whether the simulation declared any assertions at all — a simulation with no assertions also exits `0`.

code

bash · 8 lines
bash
java -cp "$CP" io.gatling.app.Gatling -s com.example.CheckoutSimulation -rf target/gatling
status=$?
case "$status" in
  0) echo "Assertions held - or the simulation declared none" ;;
  1) echo "Command line rejected - or the run never produced a verdict" ;;
  2) echo "At least one assertion failed" ;;
  *) echo "Gatling returned no status code; the process was killed or the JVM died: $status" ;;
esac

go deeper

for a junior

Memorise the three values and which one an assertion failure produces. Be ready to say that 2 means assertions failed and 0 means they held, or that none were declared.

for a middle

Be ready to explain where each code is produced: the parser returns 1, and the run-result processing folds every assertion verdict into 0 or 2 after the simulation has finished.

for a senior

Show that you know the integer is a verdict and not a report, and that failed requests do not move it by themselves. Say what you would read for detail.

for a principal

Own the contract between a load test and everything downstream. Be able to argue why a three-value signal is enough, and where a team needs a richer artefact instead.

A Gatling Community Edition run ends by handing the operating system one integer. Unless something goes looking at files afterwards, that integer is the entire result: it is what a shell `$?`, a build step, or a job runner reads to decide whether the run passed. The set of values is small, and it is declared in exactly one place — `StatusCode` in Gatling's `gatling-app` module. ## The three declared codes | code | name | what it means | |---|---|---| | `0` | `Success` | every declared assertion held — **or the simulation declared none** | | `1` | `InvalidArguments` | the command-line parser rejected the arguments it was handed | | `2` | `AssertionsFailed` | at least one assertion registered on `setUp` did not hold | There is no fourth value in that type. Anything else a runner observes did not come from `StatusCode`. ## Where each value is produced - **`2`** comes from Gatling's run-result processing. After the simulation finishes, Gatling reads the run's own log data, evaluates every assertion registered on `setUp`, prints one line per assertion with its verdict and the actual measured value, and folds the results into a single boolean. False means `AssertionsFailed`. - **`0`** comes from the same fold when the boolean is true. Note that this covers two very different situations: assertions that were checked and held, and assertions that never existed. - **`1`** comes from the argument parser, which returns `InvalidArguments` when it cannot make sense of the command line. It is narrow: it means *the command line was rejected*, not *the test went badly*. Because assertions are evaluated only once the run is over, Gatling has no in-flight gate. A threshold cannot cut a run short the moment it is breached; you get the verdict when the run ends and the data has been read back. ## Why `2`, and never `99` This is the number people carry across tools, and it is worth pinning down once: 1. **Gatling** exits `2` when an assertion fails. 2. **k6** exits `99` when a threshold is breached — a different tool, a different number, and its thresholds are re-evaluated while the run is in flight rather than only at the end. 3. **JMeter** ships no native pass/fail gate at all; a non-GUI run exits `0` even when every sample failed, so any verdict has to be computed afterwards from its result file. A build step copied from one tool's pipeline and re-pointed at another silently inverts: a script that special-cases `99` will read Gatling's `2` as "some other failure", and a script that treats any non-zero as a test failure will read Gatling's `1` — a rejected command line — as a performance regression. ## What the integer does not carry The exit code is a verdict, not a report. It does not tell you: - **which** assertion failed, or by how much — that is in the console lines Gatling prints and in the generated HTML report; - **how much** load the run generated — the request count and the rate are in the report, not in the integer; - how many requests were KO. One thing it *does* carry, against intuition: whether any request was recorded at all. With report generation on — the default — Gatling builds the report **before** it computes the status code, and the report generator refuses a run in which no request was reported, throwing instead of writing anything. That throw takes the crash path, so such a run ends at `1` and never at `0`. Read that in one direction only: `1` is shared with a rejected command line and with every other crash, so it is the `0`/`2` side that is informative — a `0` or a `2` says the log Gatling read carried at least one request, though under `--reports-only` that log belongs to an earlier run. The single way to exit `0` having sent nothing is `--no-reports` on a simulation that also declares no assertions: then Gatling never opens a log at all. That last point surprises people most often. **Failed requests do not move the exit code by themselves.** A simulation that ran to completion while every single response came back as an error exits `0` unless an assertion — something over `failedRequests` or `successfulRequests`, for instance — was registered to catch it. The requests are counted, the KO percentage is right there in the report, and the process still reports success, because nothing was asked to judge it. ## Reading the code safely in a script A short discipline covers the common mistakes: 1. **Capture the status immediately** after the Gatling process ends, before any other command overwrites it. 2. **Treat `1` as "Gatling produced no verdict"**, not as "the test failed" — the run may never have started. 3. **Do not invent meanings for other values.** If a runner reports something outside `0`, `1` and `2`, Gatling never returned a `StatusCode` at all: the process was killed or the JVM died instead — `137` for SIGKILL, `143` for SIGTERM, `130` for Ctrl+C, `134` for a JVM fatal error — or the simulation's own code called `System.exit(n)`. The codes themselves are stable across the 3.15.x line and small enough to memorise, which is exactly why interviewers ask: the number is the contract between a load test and everything downstream of it.

  • At what point in a Gatling run are the assertions evaluated?
    After the simulation has finished. Gatling reads back the run's own recorded data, evaluates every assertion registered on `setUp`, prints each verdict with its actual value, and only then maps the combined result to an exit code. There is no in-flight evaluation, so an assertion cannot abort a run the moment it is breached.
  • A Gatling run finished with 100% of its requests KO. What exit code did it return?
    `0`, unless an assertion covered that. Failed requests are counted and shown in the report, but they never move the exit code on their own — only a registered assertion turns a measurement into a verdict. An assertion over `failedRequests` is what makes a KO-heavy run exit `2`.
  • Can a Gatling run return an exit code that is not 0, 1 or 2?
    No — not from Gatling itself, and neither of the routes usually offered gets you there. A crashed run is logged and **rethrown**, so the exit call that would have carried a `StatusCode` is never reached and the JVM ends on an uncaught exception in `main` — which is `1`. A fatal `Throwable` inside an action is an explicit exit with `1`. Both land inside the set. A value outside `0`, `1` and `2` only appears when the process is killed or the JVM dies instead of returning: a signal from the environment (`137` for SIGKILL, `143` for SIGTERM, `130` for Ctrl+C, `134` for a JVM fatal error), or the simulation's own code calling `System.exit(n)`.

It is a traffic light with only three bulbs wired: green for passed, amber for "I never started", red for failed. Nothing on the pole tells you how heavy the traffic was.

saying these in an interview costs you the question

  • Thinking a failed Gatling assertion exits 1 like a generic error
  • Carrying k6's exit code 99 across to Gatling
  • Assuming exit 1 always means the command line was wrong
  • Believing KO requests alone change Gatling's exit code
  • Expecting Gatling to abort the run the moment a threshold breaks