skip to content

In a ZAP plan's env.parameters, what do failOnError, failOnWarning and continueOnFailure actually control?

level: seniorimportance: should knowfreq 42%

answer

  1. about stopping, not about reporting
  2. one question, asked between jobs
  3. continueOnFailure outranks both
  4. the exit value is computed separately

basics

~20 s

failOnError, failOnWarning and continueOnFailure control only whether a ZAP plan abandons the jobs it has not yet run. They do not set the exit value, which comes from whether any error or warning was recorded.

solid answer

~40 s

`failOnError` defaults to **true**, `failOnWarning` to **false** and `continueOnFailure` to **false**. Together they answer one question, asked between jobs: should the plan give up on the jobs it has not reached? The plan quits early when it is not set to continue on failure and either an error has been recorded with `failOnError` set, or a warning has with `failOnWarning` set. A job marked `alwaysRun: true` still runs after that decision. What these switches do **not** do is decide the outcome reported to the shell: an error means the error exit value and a warning means the warning value, regardless — so `failOnWarning: false` still ends a run on the warning value. Only the `exitStatus` job overrides that.

code

yaml · 7 lines
yaml
env:
  parameters:
    failOnError: true        # stop the remaining jobs on an error
    failOnWarning: false     # a warning does not stop them
    continueOnFailure: false # ... unless this is true, which overrides both
    progressToStdout: true
    maxDuration: 0

go deeper

for a junior

Recall the defaults — fail on an error, not on a warning, do not continue past a failure — and that they live under env.parameters, applying to the whole plan rather than to one job.

for a middle

Explain that the three switches decide only whether the remaining jobs still run, and that the exit value is computed from the recorded errors and warnings independently of them.

for a senior

Show how you would configure a plan for a pipeline: keep failing on errors, mark the reporting job alwaysRun, and treat the run's output rather than its exit value as the description of what happened.

for a principal

Own what a scanning gate is allowed to block. These switches decide how much of a run survives a fault; the policy question above them is which outcomes may stop a delivery pipeline and who may change that.

## Three switches, one question `env.parameters` carries the run-wide failure switches, and the shipped templates show their defaults: - **`failOnError: true`** - **`failOnWarning: false`** - **`continueOnFailure: false`** - (`progressToStdout: true` and `maxDuration: 0` sit beside them and are about output and time, not failure.) All three feed one predicate, evaluated between jobs: **the plan quits early when it is not set to continue on failure, and either an error has been recorded while `failOnError` is set, or a warning has been recorded while `failOnWarning` is set.** `continueOnFailure: true` therefore overrides both of the others — the plan keeps going whatever has been recorded. That predicate answers exactly one question: **do we still run the jobs we have not reached?** It is not consulted for anything else. ## What they do not control The outcome reported to the shell is computed separately, from the recorded findings alone: | what the run recorded | exit value | |---|---| | at least one error | the error value | | no error, at least one warning | the warning value | | neither | clean | No part of that reads `failOnError`, `failOnWarning` or `continueOnFailure`. Two consequences follow, and both bite in pipelines: 1. **`failOnError: false` does not buy you a green run.** It buys you a plan that keeps working after a job fails. The run still reports the error value at the end, because an error was recorded. 2. **`failOnWarning: false` — the default — does not make warnings harmless.** A warning still moves the run off a clean exit value. Any plan carrying, say, a deprecated job reports the warning value even though nothing was told to fail on a warning. The shipped template's comments are the trap here. They read `# If set exit on an error` and `# If set exit on a warning`, which invites precisely the reading the code does not support. Read them as *stop early on* rather than *exit on*. ## The gate before the first job is stricter still There is a second check, before any job runs at all, and that one is **unconditional**: if anything has already been recorded as an error while the plan was being loaded — an unknown job type, an unrecognised element in the environment, a bad job parameter — the run creates the environment and then stops, whatever the three switches say. So `failOnError: false` cannot rescue a plan that failed to load; it only affects errors raised by a job that has already run. ## `alwaysRun` and `enabled` Two per-job keys interact with all of this: - **`alwaysRun: true`** — the job runs even after the plan has decided to stop early (stopping the plan outright still overrides it). This is how a job that summarises or reports survives an earlier failure. Without it, the moment the plan decides to quit, everything below the failing job is skipped. - **`enabled: false`** — the job is skipped at run time, with an informational note. It is **not** skipped at load time: its parameters are still verified and its tests still constructed, so a disabled job can still contribute a warning or an error to the run's outcome. ## Controlling the exit value on purpose The supported way to decide the outcome is the **`exitStatus` job**, which inspects the findings the run accumulated and sets an override the exit calculation uses in preference to everything above. Because it is a job, it is subject to the same early-exit rule as every other job: if an earlier job fails and the plan decides to quit, the `exitStatus` job never runs and the default calculation applies instead. If you depend on it, mark it `alwaysRun: true` and put it at the bottom of the list. ## How to set them for a pipeline 1. Leave `failOnError: true` unless you have a specific reason. A plan that keeps attacking after its environment failed to authenticate is spending time to produce a misleading clean report. 2. Reach for `continueOnFailure: true` only when you genuinely want every job attempted and are reading the output rather than the exit value. 3. Mark the summarising or reporting job at the end of the plan `alwaysRun: true`, so a failure still produces an artefact to look at. 4. Do not treat the exit value as a description of what the run found. Unless an `exitStatus` job put it there, it describes how the plan went. A plan can end clean having scanned almost nothing, and can end on the warning value over something as inert as a deprecated job.

  • With `failOnError: false`, what exit value does a plan whose job errored report?
    The error value. The switch changed only what still ran after the failure. The exit calculation looks at whether any error was recorded and knows nothing about the switch, so the run is still reported as failed.
  • How do you make sure a reporting job runs even when an earlier job fails?
    Mark it `alwaysRun: true`. Once the plan decides to stop early, every remaining job is skipped unless it carries that key, so a summary or report placed at the bottom of the list would otherwise be the first casualty of the failure you most want to look at.
  • Does `enabled: false` stop a job affecting the run at all?
    No. It skips the work, but the job's parameters are still verified and its tests still constructed while the plan loads, so a disabled job can still put a warning or an error into the run's outcome and change the exit value.

saying these in an interview costs you the question

  • Says failOnError: false makes a failed run exit clean
  • Assumes failOnWarning: false keeps warnings out of the exit value
  • Thinks continueOnFailure only applies to warnings
  • Expects a disabled job to be entirely inert
  • Believes the exit value describes what the scan found