A Gatling build step has been exiting 0 for months on a run that judged nothing. How does that happen, and what makes Gatling return 0 in that case?

middlemust knowfreq 50%

answer

Green does not mean judged
No assertions, nothing to fail
Empty list folds to true
Reports off does not disable assertions
Console prints one line per assertion

basics

~20 s

A Gatling simulation that registers no assertions gives Gatling nothing to judge. It folds an empty list of assertion results into a single true and returns Success, exit code 0 — the same integer a genuinely passing run returns.

solid answer

~40 s

Gatling only judges a run against the assertions registered on setUp. When a simulation declares none, the run-result processing folds an empty list of assertion results into one boolean — and an empty fold starting from true stays true — so it returns Success and the process exits 0. Nothing about the traffic changes that: every request could have come back KO and the code is still 0, because no assertion was asked about it. Gatling also short-circuits harder when reports are disabled and there are no assertions: it never even reads the run's log file, and returns success immediately. The usable tell is the console — Gatling prints one line per assertion with its verdict and actual value, so a green run that printed no assertion lines judged nothing.

Gatling only judges a run against the assertions registered on `setUp`. When a simulation declares none, the run-result processing folds an empty list of assertion results into one boolean — and an empty fold starting from true stays true — so it returns `Success` and the process exits `0`. Nothing about the traffic changes that: every request could have come back KO and the code is still `0`, because no assertion was asked about it. Gatling also short-circuits harder when reports are disabled and there are no assertions: it never even reads the run's log file, and returns success immediately. The usable tell is the console — Gatling prints one line per assertion with its verdict and actual value, so a green run that printed no assertion lines judged nothing.

code

java · 18 lines

java

import io.gatling.javaapi.core.*;
import io.gatling.javaapi.http.*;

import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;

public class NoVerdictSimulation extends Simulation {

  private final HttpProtocolBuilder httpProtocol = http.baseUrl("https://example.com");

  private final ScenarioBuilder scn =
      scenario("browse").exec(http("home").get("/"));

  {
    // No assertions(...) call: this run exits 0 whatever the service does
    setUp(scn.injectOpen(atOnceUsers(50))).protocols(httpProtocol);
  }
}

go deeper

for a junior

Be ready to say that Gatling only judges what the simulation asserts, so a run with no assertions exits 0. Recognise that green does not by itself mean the service was healthy.

for a middle

Explain the mechanism: the assertion verdicts are folded into one boolean starting from true, and an empty list stays true. Mention that disabling reports does not disable assertions.

for a senior

Talk about how a suite drifts into this state — a dropped chain in a merge, a second assertions call replacing the first in the Java API — and how you would spot it in a run you did not write.

for a principal

Own the distinction between a run that passed and a run that was never judged, and be able to argue where the responsibility for closing that gap should sit.

This is the single most useful thing to understand about Gatling's exit signal, and it is a design consequence rather than a bug: a run that judged nothing and a run that judged everything and passed return the same integer.

The mechanism, step by step

A simulation registers assertions by calling assertions(...) on the object setUp returns. If it never calls it, the simulation's assertion list is empty.
When the run ends, Gatling builds a run result that carries a flag for whether the simulation had any assertions at all.
The run-result processing evaluates each assertion against the recorded data and collects the verdicts into a list.
It then folds that list into one boolean, starting from true and combining with logical and. Folding an empty list starting from true yields true. True maps to Success, which is exit code 0.

Step 4 is the whole story. There is no special case for "the user forgot to assert anything"; the empty list is vacuously satisfied, exactly as "all of no things held" is vacuously true.

The harder short-circuit

There is a second path worth knowing. Before evaluating anything, Gatling asks whether it needs the run's log data at all — it needs it if report generation is on, or if the simulation had assertions. If report generation has been turned off and the simulation declared no assertions, neither is true: Gatling never opens the log file, never computes a single statistic, and returns Success straight away.

So in that configuration the run really does exit 0 without reading a byte of its own results. Note the converse, which is the common misconception: turning reports off does not turn assertions off. With assertions declared, Gatling still parses the log and still returns 2 when one fails; it simply skips writing the HTML.

How a build step ends up here

The usual route is not carelessness so much as drift:

The simulation was written to measure first, with assertions "to be added once we know the baseline" — and the baseline conversation never happened.
A refactor moved the setUp call and the .assertions(...) chain was dropped in the merge, which compiles perfectly.
Someone added a second .assertions(...) call. In the Java API — and therefore in Kotlin and in the JavaScript/TypeScript SDK that sit on it — the second call assigns the new list over the old one, so the first set silently disappears. In a Scala simulation the two calls concatenate instead. Same-looking code, opposite outcome, and only the Scala version keeps both.

None of these produce an error, a warning, or a different exit code. The job stays green and the team reads green as "performance is fine".

What the green actually proved

Very little. It proved the process started, the simulation class loaded and instantiated, the injection profile ran to its end, and nothing crashed hard enough to kill the JVM — and, as long as reports were being generated, that at least one request was recorded, because the report generator throws rather than build a report for a run with none. It did not prove:

that any response was correct, or even successful;
that the service met any latency expectation;
that the run generated the load it was configured to generate.

A run where every request returned a 500 exits 0 on exactly the same code path.

the run	exit code	what the code proved
assertions declared, all held	`0`	every stated expectation was measured and met
assertions declared, one failed	`2`	an expectation was measured and missed
no assertions declared	`0`	only that the process finished

The first and the third row are indistinguishable to anything reading the integer, which is the whole problem.

Detecting it

Two observables are worth wiring into how you read a run:

The console. Gatling prints one line per assertion, carrying the assertion's description, its verdict, and the actual measured value. Zero such lines means zero assertions. That is a direct, no-tooling-needed check on whether a judgement happened.
The report. A generated run report shows the assertion outcomes for the run; a run with none shows none.

Both are observations, not gates — neither of them changes the exit code. Making the absence of a verdict fail is a separate design decision, and it is the interesting part of the problem.

The one-line summary to have ready

Gatling's exit code answers "did anything I asked about turn out badly?" It does not answer "did I ask about anything?" Those are different questions, and only the first one is wired to the integer a build step reads.

This is the single most useful thing to understand about Gatling's exit signal, and it is a design consequence rather than a bug: **a run that judged nothing and a run that judged everything and passed return the same integer.** ## The mechanism, step by step 1. A simulation registers assertions by calling `assertions(...)` on the object `setUp` returns. If it never calls it, the simulation's assertion list is empty. 2. When the run ends, Gatling builds a run result that carries a flag for whether the simulation had any assertions at all. 3. The run-result processing evaluates each assertion against the recorded data and collects the verdicts into a list. 4. It then folds that list into one boolean, starting from `true` and combining with logical *and*. **Folding an empty list starting from `true` yields `true`.** True maps to `Success`, which is exit code `0`. Step 4 is the whole story. There is no special case for "the user forgot to assert anything"; the empty list is vacuously satisfied, exactly as "all of no things held" is vacuously true. ## The harder short-circuit There is a second path worth knowing. Before evaluating anything, Gatling asks whether it needs the run's log data at all — it needs it if report generation is on, or if the simulation had assertions. If report generation has been turned off **and** the simulation declared no assertions, neither is true: Gatling never opens the log file, never computes a single statistic, and returns `Success` straight away. So in that configuration the run really does exit `0` without reading a byte of its own results. Note the converse, which is the common misconception: **turning reports off does not turn assertions off.** With assertions declared, Gatling still parses the log and still returns `2` when one fails; it simply skips writing the HTML. ## How a build step ends up here The usual route is not carelessness so much as drift: - The simulation was written to *measure* first, with assertions "to be added once we know the baseline" — and the baseline conversation never happened. - A refactor moved the `setUp` call and the `.assertions(...)` chain was dropped in the merge, which compiles perfectly. - Someone added a second `.assertions(...)` call. In the **Java API** — and therefore in Kotlin and in the JavaScript/TypeScript SDK that sit on it — the second call **assigns** the new list over the old one, so the first set silently disappears. In a **Scala** simulation the two calls concatenate instead. Same-looking code, opposite outcome, and only the Scala version keeps both. None of these produce an error, a warning, or a different exit code. The job stays green and the team reads green as "performance is fine". ## What the green actually proved Very little. It proved the process started, the simulation class loaded and instantiated, the injection profile ran to its end, and nothing crashed hard enough to kill the JVM — and, as long as reports were being generated, that at least one request was recorded, because the report generator throws rather than build a report for a run with none. It did **not** prove: - that any response was correct, or even successful; - that the service met any latency expectation; - that the run generated the load it was configured to generate. A run where every request returned a 500 exits `0` on exactly the same code path. | the run | exit code | what the code proved | |---|---|---| | assertions declared, all held | `0` | every stated expectation was measured and met | | assertions declared, one failed | `2` | an expectation was measured and missed | | **no assertions declared** | `0` | **only that the process finished** | The first and the third row are indistinguishable to anything reading the integer, which is the whole problem. ## Detecting it Two observables are worth wiring into how you read a run: - **The console.** Gatling prints one line per assertion, carrying the assertion's description, its verdict, and the actual measured value. **Zero such lines means zero assertions.** That is a direct, no-tooling-needed check on whether a judgement happened. - **The report.** A generated run report shows the assertion outcomes for the run; a run with none shows none. Both are observations, not gates — neither of them changes the exit code. Making the absence of a verdict *fail* is a separate design decision, and it is the interesting part of the problem. ## The one-line summary to have ready Gatling's exit code answers "did anything I asked about turn out badly?" It does not answer "did I ask about anything?" Those are different questions, and only the first one is wired to the integer a build step reads.

Does passing the no-reports option to a Gatling run also switch off its assertions?
No. Disabling reports only stops the HTML from being written. If the simulation declared assertions, Gatling still reads the run's log, evaluates them and returns `2` when one fails. The two only interact when there are also no assertions: then Gatling skips reading the log entirely and returns success immediately.
In Gatling's Java DSL, what happens if a simulation calls .assertions(...) twice on the same setUp?
The second call replaces the first. The Java API assigns the new list over the stored one, so only the assertions in the last call are ever evaluated — and Kotlin and the JavaScript/TypeScript SDK inherit that because they sit on the Java API. A Scala simulation concatenates instead, keeping both sets.
Without changing the pipeline, what is the quickest way to tell whether a green Gatling run judged anything?
Look at the console output of the run. Gatling prints one line per assertion with its description, verdict and actual measured value. If the run printed none, the simulation registered none, and the `0` means nothing was asked rather than everything passed.

It is a spelling checker run on an empty document. It reports no mistakes, which is perfectly true and tells you nothing whatsoever about your writing.

saying these in an interview costs you the question

Reading a green Gatling run as proof the service performed well
Assuming Gatling fails a run that declares no assertions
Thinking disabling reports also disables assertion evaluation
Believing a second assertions call in Java adds to the first
Expecting KO responses to fail a run with no assertions

A Gatling build step has been exiting 0 for months on a run that judged nothing. How does that happen, and what makes Gatling return 0 in that case?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

The mechanism, step by step

The harder short-circuit

How a build step ends up here

What the green actually proved

Detecting it

The one-line summary to have ready

▾common follow-ups3 questions

▾analogy~10 s

▾red flags5 flags

The mechanism, step by step

The harder short-circuit

How a build step ends up here

What the green actually proved

Detecting it

The one-line summary to have ready

answer

go deeper