skip to content

What do the exit codes of ZAP's packaged baseline scan script mean, and what does its -I flag change?

level: juniorimportance: must knowfreq 55%

answer

  1. four codes, checked in a fixed order
  2. fail beats warn beats pass
  3. no separate code for an error
  4. -I deletes one branch, not the printing

basics

~20 s

zap-baseline.py exits 1 if a rule marked FAIL alerted, 2 if only WARN-bucket rules did, 0 if at least one rule ran and raised nothing, and 3 for everything else. -I removes only the exit-2 branch.

solid answer

~50 s

ZAP's packaged scan scripts share one exit ladder, walked in order: an exception exits `3`; a non-empty `FAIL` bucket exits `1`; a non-empty `WARN` bucket with nothing failing exits `2`; at least one rule that ran and raised nothing exits `0`; anything left exits `3`. Which bucket a finding lands in comes from the tab-separated rule file supplied with `-c` or `-u`, where each line pins one rule id to `IGNORE`, `INFO`, `WARN` or `FAIL`; a rule the file never mentions defaults to `WARN`. There is no separate error code, so a crash, an unreadable flag, an unreachable target and a run that recorded no URLs all land on `3`. `-I` does not silence warnings -- they are still raised, printed and counted -- it deletes the `2` branch only, and a rule marked `FAIL` still exits `1` with `-I` set.

code

bash · 9 lines
bash
zap-baseline.py -t https://example.com -c rules.conf

# the four codes this script can return
case $? in
  0) echo "a rule ran and raised nothing; no gating bucket had anything" ;;
  1) echo "a rule marked FAIL in rules.conf alerted" ;;
  2) echo "a WARN-bucket rule alerted and nothing marked FAIL did" ;;
  3) echo "anything else: crash, unreadable option, or no URL reached" ;;
esac

go deeper

for a junior

Learn the four codes and their order. The one to memorise is that a crash and a run that reached nothing share the same code, so a non-zero result is not automatically a security finding.

for a middle

Be able to say where each bucket comes from: the tab-separated rule file behind -c or -u, and the WARN default for any rule the file does not mention. Explain -I as deleting one branch, not as quietening output.

for a senior

Show that you read the printed summary line and URL count beside the code, because the exit value alone cannot separate a broken run from a clean one. Say what you would alert on and what you would only log.

for a principal

The judgment worth owning is which of these codes your organisation lets stop a release, and what evidence a team must attach when it does not. Treat the crowded code as a reliability signal about the scan itself.

## One ladder, four codes, walked in order ZAP's packaged scan scripts -- `zap-baseline.py`, `zap-full-scan.py` and `zap-api-scan.py` -- all end the same way. By the time the run finishes, every finding it kept has been sorted into a bucket named after a **verdict**, and the script then walks a short chain of branches and exits on the first one that matches. | exit code | the branch that fired | what it actually tells you | |---|---|---| | `3` | the run raised an exception | start-up failed, an option was unreadable, or the target could not be opened at all | | `1` | the `FAIL` bucket is not empty | a rule you deliberately pinned to `FAIL` alerted | | `2` | the `WARN` bucket is not empty and `FAIL` is empty | something alerted that you marked `WARN` or never classified | | `0` | at least one rule ran and raised nothing | the run reached something, and no gating bucket had anything in it | | `3` | nothing above matched | the run recorded no URLs, so no rule ever got the chance to pass | `3` appears at both ends of that chain on purpose. It is the code for "the run broke" **and** the code for "the run worked and had nothing to score". ## Where the buckets come from - `-c <file>` reads a tab-separated rule file from disk; `-u <url>` fetches the same file over HTTP. - Each line pins one rule id to `IGNORE`, `INFO`, `WARN` or `FAIL`. - A rule the file never mentions defaults to **`WARN`**, so the warning bucket is where everything unclassified accumulates. The `-i` flag flips that default to `INFO`. - `-g <file>` writes a starting file with **every** rule already set to `WARN`, which is exactly why an ungoverned run so reliably exits `2`. - Run with no rule file at all and every rule is unlisted, therefore `WARN`, therefore the script exits `2` the moment anything alerts. ## What `-I` changes, and what it does not `-I` is the most commonly misread flag in this family, because its name suggests it quietens the output. It does not. 1. It deletes the exit-`2` branch, and nothing else. 2. Warnings are still raised, still printed with their rule names, and still counted in the summary line the run prints. 3. A rule marked `FAIL` still exits `1` when `-I` is set -- the flag is tested only on the warning branch. 4. With `-I` set and only warnings raised, the run falls through to the branch below: `0` if at least one rule ran clean, `3` if none did. So `-I` turns "warnings are blocking" into "warnings are advisory". It is not a way to hide them, and it is not a way to soften a `FAIL`. ## Why `3` is the crowded code There is no distinct error code in this contract, so `3` collects several unrelated situations: - an unhandled exception anywhere in the run; - a target ZAP could not open; - a file-based option pointing at something the script cannot read; - a run that recorded **no URLs**, which skips the whole counting block and leaves the pass count at zero; - on `zap-baseline.py`'s automation-framework path, a summary file the script could not find or read afterwards. That is the practical reason a pipeline should not treat the exit code as the whole result. "The tool crashed" and "the tool ran and reached nothing" are the same number. ## What the run prints beside the code Every one of these scripts prints a single tab-separated summary line before it exits, naming each bucket and its size: failures and warnings are each split into a **new** count and an **in-progress** count, followed by the informational, ignored and passing counts. - Only the **new** counters feed the exit ladder. - The in-progress counters come from a separate JSON file supplied with `-p` and are printed but never gated on. - The passing count is a count of **rules that ran and raised nothing** -- not of URLs visited or requests sent. Reading that line beside the exit code is what turns `2` from a number into a diagnosis: it tells you how many distinct rules alerted, how many you had already classified, and whether the run saw enough of the target for any of it to mean something. ## The one-line summary for a pipeline `1` is the only code that means "a rule you marked as blocking alerted". `2` means the run found something you had not classified. `0` means the gate passed. `3` always deserves the log.

  • With -I set and only warnings raised, which code comes back?
    `-I` removes the exit-`2` branch, so the run falls through to the next test. If at least one rule ran and raised nothing, you get `0`; if the run recorded no URLs and nothing could pass, you get `3` instead. The warnings are still printed either way.
  • Do the full scan and API scan scripts use a different ladder?
    No -- all three packaged scripts walk the same four branches in the same order, and all three read the same tab-separated rule file. What differs is which rules can land in a bucket at all, since each script runs a different set of scan rules against the target.
  • Why does a run with no rule file so often exit 2?
    A rule the file never mentions defaults to `WARN`, and with no file supplied every rule is unmentioned. So the first alert of any kind fills the warning bucket and the run exits `2`. Supplying a file, or `-I`, is what changes that.

saying these in an interview costs you the question

  • Says exit 3 means the scan found serious issues
  • Thinks -I stops warnings being printed at all
  • Believes -I also stops a FAIL-marked rule failing the run
  • Assumes any non-zero exit means findings were raised
  • Expects a distinct exit code for a crash or bad flag