In a Newman command line, what does `--bail` change about how a collection run proceeds?
answer
- One boolean handed to the runner
- A bare flag and one modifier agree
- Assertion failures count, not only thrown errors
- Stopping early is not passing
- The report covers only what ran
basics
~20 sA bare --bail, or --bail failure, makes Newman turn on the runtime's stopOnFailure option, which ends the whole run at the first failure instead of continuing through the remaining requests. It does not change the exit code.
solid answer
~40 s`--bail [modifiers]` accepts an optional comma-separated modifier list. When it is given bare, or with the `failure` modifier, Newman sets the runtime's `stopOnFailure` run option to `true`; otherwise it stays `false`. With `stopOnFailure` on, the runtime watches assertion events during script execution and turns a failed assertion into an execution error, so a failing assertion — not just a thrown script error — ends the run there. The remaining requests are never sent, and the summary and reports therefore cover only the items that ran. `--bail` and the exit code are independent concerns: the failure is still recorded, so the command still exits 1 unless `-x` is also given. Bailing shortens a run; it never makes one pass.
code
bash · 2 linesnewman run ./collection.json --bail
echo "stopped early, exit status was $?"go deeper
Know that the flag makes the run stop at the first failure instead of continuing to the end of the collection, and that stopping early is not the same as the run passing.
Explain that Newman translates the flag into a single boolean run option for the runtime, and that with it on a non-passing assertion — not just a thrown error — becomes the error that ends the run.
Show judgment about when a truncated run helps and when it hides things: dependent steps and expensive targets favour bailing, regression sweeps and flaky suites do not, and bailed reports must not feed failure trend lines.
Own the split between a fast gate and a complete sweep. Decide which runs bail, which run to completion, and how the team reads reports that are deliberately partial without drawing false conclusions.
## What the flag sets `--bail [modifiers]` is declared on the Newman CLI as an option taking an optional value, which is parsed as a comma-separated list. Newman then normalises it: an array of modifiers becomes an object with each modifier as a key set to `true`. The decision it drives is a single boolean: - a **bare `--bail`** — the option present with no value — sets the runtime's `stopOnFailure` to `true`; - **`--bail failure`** sets the same thing, because the `failure` key is what is checked; - **omitting the flag entirely** leaves `stopOnFailure` at `false`. `stopOnFailure` is a run option on the runtime that Newman drives, not a flag the runtime invented for Newman. Newman's job here is purely translation: parse the option, produce the boolean, hand it to the runner. ## What actually trips a bail This is the part worth knowing, because "stops on failure" is vaguer than the mechanism. With `stopOnFailure` off, a failed assertion is recorded and the run carries on to the next item; the runner only aborts on things it considers execution errors. With `stopOnFailure` on, the runtime additionally **subscribes to assertion events for the executing script** and collects the names of assertions that did not pass. When the script finishes, those collected failures are folded into the execution result as an error. The item command then sees an execution error and stops the run. So the practical answer is that all of these end a bailing run: - an assertion that did not pass, in a pre-request or test script; - a script that threw before its assertions ran; - a request that could not be sent at all. And the run stops *between* items: the current item finishes its own chain, and no further item is started. ## Bail and the exit code are independent These two get conflated constantly, so hold them apart: | Question | Answer | |---|---| | Does `--bail` change the exit code? | No. The failure is still in the summary, so the command still exits 1. | | Does `--bail` reduce the number of failures reported? | Yes — later items never run, so their failures never exist. | | Does `-x` stop the run early? | No. It only suppresses the exit code. | | Can both be used together? | Yes: the run stops early *and* reports 0. That combination is rarely what anyone wants. | The cleanest way to say it: `--bail` decides **how much of the collection runs**, and `-x` decides **what the process tells its caller**. Neither one implies the other. ## What the report looks like afterwards Because the run genuinely stops, the summary reflects a partial run: 1. The statistics under `summary.run.stats` count only the items, requests, scripts and assertions that actually executed. 2. `summary.run.failures` holds the one failure that caused the stop, plus anything that failed earlier. 3. Any file-writing reporter still runs and still writes — a bailed run produces a report, just a shorter one. That matters when a report is being read as a picture of suite health. A bailed run says "something broke here"; it does not say how much else is broken, and a trend line built from bailed runs will systematically understate failure counts. ## When bailing is the right call - **Fast feedback on a run whose steps depend on each other.** When request three cannot possibly succeed if request two failed, finishing the collection wastes time and floods the report with derived noise. - **Expensive or rate-limited targets**, where continuing after a genuine break costs real quota or real money. - **A smoke run whose only question is "is anything broken at all".** The first failure is the whole answer. And when it is the wrong call: - **A regression run whose value is the complete failure list.** Bailing throws away exactly the information you ran it for. - **A flaky suite.** One unstable early item will mask every later result, run after run, and the picture never improves. - **Any run whose report feeds a dashboard of counts.** Partial runs make the counts meaningless. ## Getting it right in practice Decide per invocation rather than per repository. A collection can be run twice with different flags — bail on the pull-request check where speed wins, run to completion on the nightly where the full list wins — because the flag lives on the command, not in the collection document. And when you do bail, read the summary as what it is: a run that was cut short deliberately, not a measurement of how healthy the suite is.
- Does `--bail` make a failing run exit with 0 because it stopped deliberately?No. The failure that caused the stop is recorded in `summary.run.failures` exactly like any other, so the command still exits 1. Only `-x` suppresses the exit code, and it does so regardless of whether the run bailed. Stopping early and reporting success are unrelated decisions.
- With `--bail` in effect, does a failed assertion stop the run, or only a thrown script error?Both. When the option is on, the runtime subscribes to assertion events during script execution and folds any non-passing assertion into the execution result as an error, which is what stops the run. Without the option, that subscription is not set up and a failed assertion is merely recorded.
- What does a bailed run's JUnit or JSON report contain?Only the items that actually executed. The reporters still run and still write their files, but the statistics count just the executed items, requests and assertions, and the failures list holds the one that stopped the run. Treat it as a truncated sample, never as a suite-wide failure count.
saying these in an interview costs you the question
- Thinks bailing makes the command exit zero
- Believes only thrown script errors trip a bail
- Uses a bailed run's counts as suite-wide failure totals
- Confuses stopping the run with suppressing the exit code
- Bails a flaky suite and never sees the later failures
- Assumes the flag is stored in the collection document