Why does Cypress's `cypress.run()` resolve instead of rejecting when tests fail?

seniornice to knowfreq 26%

answer

Failures are data, not exceptions
Three outcomes, one rejection
A results object, not a boolean
Nobody sets your process status
Rejection means it never got started

basics

~20 s

Because a failing test is a normal outcome of running a suite, not an error in running it. cypress.run() resolves with a results object carrying totalFailed and per-spec stats; it rejects only when Cypress could not run at all, such as an uninstalled binary.

solid answer

~40 s

Failures are data, not exceptions. require('cypress').run() resolves with a results object whatever the verdict — totalTests, totalPassed, totalFailed, totalDuration, and a runs[] entry per spec with its stats, tests[], screenshots[], video and error. Rejection is reserved for the case where Cypress could not run at all: a binary that was never installed, a missing module dependency. There is a third outcome to handle — Cypress started but the tests could not — which resolves with { failures, message }. Because the promise resolves, nothing sets your process's exit status for you: a script that logs the results and ends exits 0 even for a red suite, so you must call process.exit() yourself with result.totalFailed, or 1, as you prefer.

Failures are data, not exceptions. `require('cypress').run()` resolves with a results object whatever the verdict — `totalTests`, `totalPassed`, `totalFailed`, `totalDuration`, and a `runs[]` entry per spec with its `stats`, `tests[]`, `screenshots[]`, `video` and `error`. Rejection is reserved for the case where Cypress could not run at all: a binary that was never installed, a missing module dependency. There is a third outcome to handle — Cypress started but the tests could not — which resolves with `{ failures, message }`. Because the promise resolves, **nothing sets your process's exit status for you**: a script that logs the results and ends exits `0` even for a red suite, so you must call `process.exit()` yourself with `result.totalFailed`, or `1`, as you prefer.

code

javascript · 21 lines

javascript

const cypress = require('cypress')

cypress
  .run({
    spec: 'packages/checkout/cypress/e2e/**/*.cy.js',
    browser: 'chrome',
    reporter: 'junit',
    reporterOptions: { mochaFile: 'reports/checkout-[hash].xml' },
  })
  .then((result) => {
    if (result.failures) {
      console.error(result.message)
      process.exit(result.failures)
    }
    console.log(`${result.totalFailed} of ${result.totalTests} failed`)
    process.exit(result.totalFailed)
  })
  .catch((err) => {
    console.error(err.message)
    process.exit(1)
  })

go deeper

for a junior

Know that Cypress can be invoked from Node as well as from the command line, and that the Node call hands you the results as an object rather than as an exit status.

for a middle

Explain the three outcomes and why only one rejects, and name the fields on the results object you would actually read.

for a senior

Show that you know you have taken over the exit contract: what your script exits with in each case, and how that interacts with a pipeline that treats non-zero as failure.

for a principal

Decide when a team is allowed to leave the CLI at all, given that a bespoke Node entry point becomes code somebody has to keep correct as the runner changes.

Failures are data, not errors

require('cypress').run(options) returns a Promise. That promise resolves with a results object even when tests fail, because a failing test is a normal, expected outcome of running a suite, not an error in running it. Reserving rejection for "a test failed" would make the common case an exception and force every caller into a catch block.

There are three outcomes, and only one of them rejects:

Cypress ran the tests. The promise resolves with the results object, whatever the verdict.
Cypress ran, but the tests could not start. The promise resolves with a small object of the shape { failures, message } — a non-zero failures count and an explanatory message.
Cypress could not run at all — the binary is not installed, a module dependency is missing. Only here does the promise reject, with a detailed error.

So a correct script checks all three: catch for case 3, a result.failures check for case 2, and result.totalFailed for case 1.

What the resolved object carries

The results object is the reason to use the Module API at all. It includes, among other fields:

totalTests, totalPassed, totalFailed, totalPending, totalSkipped, totalDuration.
runs[], one entry per spec, each with its own stats, tests[] (title, state, displayError, per-attempt state), screenshots[], video path, reporterStats, and an error field that is set when the spec itself errored.
runUrl when the run was recorded, plus cypressVersion, browserName, browserVersion, osName and the resolved config.

Options mirror the CLI in camelCase: spec, browser, headed, reporter, reporterOptions, config, env, expose, record, key, group, tag, testingType, posixExitCodes, quiet, project. The same module also exposes cypress.open() for the interactive side.

Your script owns the exit code

This is the part people get wrong. Nothing sets the Node process's exit status for you. If your script resolves, logs the results and falls off the end, the process exits 0 — and a red suite reports green to whatever called the script. You must call process.exit() yourself, deciding what the status should be:

process.exit(result.totalFailed) reproduces the CLI's default tally behaviour.
process.exit(result.totalFailed > 0 ? 1 : 0) gives you POSIX semantics without the flag.
process.exit(result.failures) for the "tests could not start" case, before you look at totalFailed at all.

When the Module API earns its keep

The CLI is the right default; the Module API pays off when you need the numbers as values rather than as an exit status:

Posting a summary, or the failing tests' displayError strings, somewhere a human will read it.
Rerunning exactly the specs that failed, taken from runs[].spec and runs[].stats.failures.
Composing several runs in one process — different --spec sets against different browsers — and reducing their results into one verdict.
Any pipeline running with --posix-exit-codes / posixExitCodes: true, where the exit status no longer carries the count and something has to.

The trade is that you have taken over responsibility for the exit contract, the error handling and the reporting that the CLI gave you for free. Do it when you are adding something, not to reimplement the CLI in JavaScript.

Shape and siblings

Two details save time when you first write one of these scripts:

The options are camelCase, not flags. spec, browser, headed, reporter, reporterOptions, configFile, posixExitCodes, testingType. There are no leading dashes and no string parsing; config, env, expose and reporterOptions take real objects.
async/await works. const results = await cypress.run() reads better than the .then() chain, but it moves case 3 into a try/catch — the two resolution cases still have to be distinguished inside the try, and it is easy to forget the { failures, message } shape once the catch looks like it is handling everything.

The same module exports cypress.open(), which launches the interactive app from Node with a smaller option set — browser, config, configFile, detached, env, expose, global, port, project, testingType. It is rarely worth scripting, for the same reason cypress open takes few flags: there is a human in front of it who can make those choices.

## Failures are data, not errors `require('cypress').run(options)` returns a `Promise`. That promise resolves with a results object **even when tests fail**, because a failing test is a normal, expected outcome of running a suite, not an error in running it. Reserving rejection for "a test failed" would make the common case an exception and force every caller into a `catch` block. There are three outcomes, and only one of them rejects: 1. **Cypress ran the tests.** The promise resolves with the results object, whatever the verdict. 2. **Cypress ran, but the tests could not start.** The promise resolves with a small object of the shape `{ failures, message }` — a non-zero `failures` count and an explanatory message. 3. **Cypress could not run at all** — the binary is not installed, a module dependency is missing. Only here does the promise reject, with a detailed error. So a correct script checks all three: `catch` for case 3, a `result.failures` check for case 2, and `result.totalFailed` for case 1. ## What the resolved object carries The results object is the reason to use the Module API at all. It includes, among other fields: - `totalTests`, `totalPassed`, `totalFailed`, `totalPending`, `totalSkipped`, `totalDuration`. - `runs[]`, one entry per spec, each with its own `stats`, `tests[]` (title, state, `displayError`, per-attempt state), `screenshots[]`, `video` path, `reporterStats`, and an `error` field that is set when the spec itself errored. - `runUrl` when the run was recorded, plus `cypressVersion`, `browserName`, `browserVersion`, `osName` and the resolved `config`. Options mirror the CLI in camelCase: `spec`, `browser`, `headed`, `reporter`, `reporterOptions`, `config`, `env`, `expose`, `record`, `key`, `group`, `tag`, `testingType`, `posixExitCodes`, `quiet`, `project`. The same module also exposes `cypress.open()` for the interactive side. ## Your script owns the exit code This is the part people get wrong. Nothing sets the Node process's exit status for you. If your script resolves, logs the results and falls off the end, the process exits `0` — and a red suite reports green to whatever called the script. You must call `process.exit()` yourself, deciding what the status should be: - `process.exit(result.totalFailed)` reproduces the CLI's default tally behaviour. - `process.exit(result.totalFailed > 0 ? 1 : 0)` gives you POSIX semantics without the flag. - `process.exit(result.failures)` for the "tests could not start" case, before you look at `totalFailed` at all. ## When the Module API earns its keep The CLI is the right default; the Module API pays off when you need the numbers as values rather than as an exit status: - Posting a summary, or the failing tests' `displayError` strings, somewhere a human will read it. - Rerunning exactly the specs that failed, taken from `runs[].spec` and `runs[].stats.failures`. - Composing several runs in one process — different `--spec` sets against different browsers — and reducing their results into one verdict. - Any pipeline running with `--posix-exit-codes` / `posixExitCodes: true`, where the exit status no longer carries the count and something has to. The trade is that you have taken over responsibility for the exit contract, the error handling and the reporting that the CLI gave you for free. Do it when you are adding something, not to reimplement the CLI in JavaScript. ## Shape and siblings Two details save time when you first write one of these scripts: - **The options are camelCase, not flags.** `spec`, `browser`, `headed`, `reporter`, `reporterOptions`, `configFile`, `posixExitCodes`, `testingType`. There are no leading dashes and no string parsing; `config`, `env`, `expose` and `reporterOptions` take real objects. - **`async`/`await` works.** `const results = await cypress.run()` reads better than the `.then()` chain, but it moves case 3 into a `try`/`catch` — the two resolution cases still have to be distinguished inside the `try`, and it is easy to forget the `{ failures, message }` shape once the `catch` looks like it is handling everything. The same module exports `cypress.open()`, which launches the interactive app from Node with a smaller option set — `browser`, `config`, `configFile`, `detached`, `env`, `expose`, `global`, `port`, `project`, `testingType`. It is rarely worth scripting, for the same reason `cypress open` takes few flags: there is a human in front of it who can make those choices.

What does a resolved `{ failures, message }` object from `cypress.run()` mean?
It is the third outcome: Cypress itself ran, but the tests could not start. `failures` is a non-zero number and `message` explains why. Check it before reading `totalFailed`, because a results object for a real run will not have that shape and the two cases deserve different reporting.
When is the `cypress run` CLI the better choice over `cypress.run()`?
Almost always. The CLI already owns the exit contract, the reporter wiring and the error handling. Reach for the Module API only when you need the numbers as values — composing several runs, rerunning exactly what failed, or reporting results somewhere — not to reimplement the CLI in JavaScript.

saying these in an interview costs you the question

Expects cypress.run() to reject when a test fails
Wraps only a catch block and never checks totalFailed
Assumes the Node process exits non-zero automatically
Ignores the resolved {failures, message} could-not-start case
Rebuilds the whole CLI in a script for no added value

Why does Cypress's `cypress.run()` resolve instead of rejecting when tests fail?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

Failures are data, not errors

What the resolved object carries

Your script owns the exit code

When the Module API earns its keep

Shape and siblings

▾common follow-ups2 questions

▾red flags5 flags

Failures are data, not errors

What the resolved object carries

Your script owns the exit code

When the Module API earns its keep

Shape and siblings

answer

go deeper