skip to content

In a parallel Cypress run, what does `--auto-cancel-after-failures 3` actually stop?

level: middleimportance: nice to knowfreq 28%

answer

  1. Counted across the whole run
  2. Failed tests, not failed specs
  3. In-flight work is never killed
  4. Flaky results are excluded
  5. false turns it off for one run

basics

~10 s

Once three tests have failed anywhere in the run, Cypress Cloud stops handing out specs and marks the run canceled. Specs already running finish and report; every spec not yet started is reported skipped.

solid answer

~50 s

`cypress run --auto-cancel-after-failures 3` overrides the Cypress Cloud project's Auto Cancellation threshold for that run. The Cloud counts **failed tests across every machine in the run**, not per machine and not per spec; when the count reaches three it stops handing out new specs and marks the run canceled. Work already in flight is not killed — a machine mid-spec finishes it and reports its results — but no machine picks up another spec, and each remaining spec appears in the CI output saying it was skipped because the run has been canceled. **Flaky tests never count**: a test that fails an attempt and then passes is recorded as flaky, not failed. Passing `false` instead of a number disables cancellation for that run, and the first value any machine sends for a run is the one that applies.

code

bash · 5 lines
bash
# branch runs: stop at the first failure
npx cypress run --record --parallel --auto-cancel-after-failures 1

# release runs: never cut the suite short
npx cypress run --record --parallel --auto-cancel-after-failures false

go deeper

for a junior

Know that Auto Cancellation is a Cypress Cloud feature that stops a recorded run once enough tests have failed, and that the CLI flag overrides the project's threshold.

for a middle

Explain precisely what the number counts — failed tests across the whole run — and what still happens afterwards: in-flight specs finish, nothing new starts, the rest report as skipped.

for a senior

Show that you read a canceled run as a partial sample: the failures present are the ones that tripped it plus whatever was in flight, and the specs never started are unknown, not passing.

for a principal

Decide where the threshold lives and who may override it, and whether the machine time saved is worth the incomplete failure picture on the runs your organisation reports from.

## What the threshold counts `cypress run --auto-cancel-after-failures 3` sets the Auto Cancellation threshold for that run only, overriding whatever the Cypress Cloud project setting says. The number counts **failed tests across the whole run**, aggregated over every machine — not per machine, not per spec, and not failed specs. Three different machines each reporting one failed test reaches a threshold of 3 exactly as a single machine reporting three does. Two exclusions matter: - **Flaky tests never count.** A test that fails an attempt and then passes is recorded as flaky, not failed, so it contributes nothing to the threshold. - **Non-test failures** — a spec that could not run at all, for example — are not the failing tests the threshold is counting. ## What stops, and what does not When the count reaches the threshold, Cypress Cloud stops handing out specs and marks the run canceled. That is a *cancellation of the run*, not a kill signal to the machines: - A machine that is **mid-spec finishes that spec** and reports its results in full. - **No machine starts another spec.** Each remaining spec is reported in the CI output in place of its results — "This spec and its tests were skipped because the run has been canceled" — and appears as SKIPPED in the run summary table. - The cancellation applies to **every machine in the run**, not only the one that reported the failure that crossed the line. - The Cloud reports what it saved: a banner on the run naming the failure count that triggered it, how many of the total specs were skipped, and the CI time not spent. ## The flag against the project setting | value passed | effect on that run | |---|---| | `--auto-cancel-after-failures 3` | cancel once three tests have failed | | `--auto-cancel-after-failures false` | Auto Cancellation off for this run | | flag omitted | fall back to the project's configured threshold | The project setting is the default and the flag is the per-run override, which is what lets one pipeline fail fast on branch runs and let release runs go all the way through without anyone editing project settings between builds. The default threshold in Cypress Cloud is **1 failure**. Inside a single run the setting is decided once: the first value any machine sends wins, and a machine arriving later with a different `--auto-cancel-after-failures` value is rejected with a mismatch error telling you the first setting takes precedence. In a parallel build, that means the value has to be the same in every job that joins the run, exactly like the build id. ## Finding what tripped it The run header carries counts of failed, flaky, skipped, pending and passed tests, and clicking the failed count opens the Test Results tab filtered to those failures — the shortest path to the tests that crossed the threshold. The Tests for Review panel on the Overview tab orders results failures-first and reaches the same handful. Both matter more than usual on a canceled run, because the CI log is mostly skipped specs by then and the few real failures are easy to lose in it. ## Reading a run that was canceled The important habit is remembering what a canceled run is **not**: it is not a complete picture of the suite. Its results cover the specs that had already started, so the failures you can see are the ones that crossed the threshold plus whatever the in-flight specs happened to report — not necessarily every failure a full run would have produced. Practical consequences: 1. Fix what you see, but do not conclude that nothing else is broken; the next run may surface more. 2. Reports that score what the suite exercised are computed only over the specs that actually ran, so a canceled run's coverage-style numbers are a partial sample and are not comparable with a complete run's. 3. Test-level analytics still count the partial results, so a run that cancelled after a couple of failures still moves failure-rate aggregates. A canceled run is recorded with a terminal status of `cancelled`, and that status is what flows out to status checks and notifications. Those integrations do not distinguish an automatic cancellation from someone pressing cancel in the Cloud — the run header and the run's Properties tab do, showing Auto Cancellation as Applied along with the threshold that was in force.

  • Two Cypress machines in one run pass different --auto-cancel-after-failures values. What happens?
    The first value any machine sends for the run takes precedence, and the machine arriving with a different value is rejected with a mismatch error naming both the flag and the existing run. Like the build id, the cancellation threshold is decided once per run, so every job that joins has to pass the same value — or none at all, and inherit the project setting.
  • Why can the failures visible in an auto-canceled Cypress run understate what is broken?
    Cancellation stops new specs from starting, so the run only ever contains results for specs that had already begun. What you see is the failures that crossed the threshold plus whatever the in-flight specs reported — not every failure a complete run would have produced. Fix those, but do not read a canceled run as evidence that nothing else fails.

saying these in an interview costs you the question

  • Thinks the threshold counts failures per machine
  • Believes running specs are killed mid-test
  • Counts flaky tests toward the cancellation threshold
  • Reads a canceled run as the full failure picture
  • Expects the flag to work without recording