When a ZAP packaged scan exits 0 in your pipeline, what has that actually established about the run?

## What the code is actually asserting ZAP's packaged scan scripts derive their exit code from a short chain of branches, tested in order. Exit `0` is not the first branch and not a special case; it is the one reached when everything above it has failed to match. Getting there requires all of the following at the same time: 1. The run finished without raising -- no unreadable option, no target the scan could not open. 2. The run recorded **at least one URL**. If it recorded none, the whole counting block is skipped. 3. At least one scan rule ran and raised nothing, giving a non-zero passing count. 4. The failing bucket was empty. 5. The warning bucket was empty, or `-I` removed that branch. Each of those is worth saying out loud, because only the first two are about the scan reaching anything, and the second one is much weaker than it sounds. ## The passing count is a count of rules, not of coverage This is the sentence that changes how people read a green ZAP stage. The passing count is built by walking the rules the run knows about and keeping the ones that appear in no finding. A rule that was never given anything interesting to look at is indistinguishable, in that count, from a rule that examined the whole application and was satisfied. So a large passing count means "many rules had no complaint", not "many pages were checked". Combine that with the URL guard -- one URL is enough -- and the failure mode is clear: - the target is reachable, so the run opens it and records that one URL; - the crawl finds nothing beyond it (a single-page shell, a login wall, a robots-driven dead end, a scope rule that excluded the application); - every rule sees only that one response, raises nothing, and lands in the passing count; - the failing and warning buckets stay empty; - the run exits `0`. Nothing in that chain is a bug. The script did what it was asked. But the pipeline now has a green check that means "one URL was fetched and nobody complained". ## The one case that does go red If the run records **no** URLs, the script prints a message asking whether the target was accessible, skips the counting block entirely, and leaves the passing count at zero -- which drops it past the `0` branch to the catch-all code `3`. On `zap-baseline.py`'s automation-framework path the same situation shows up differently: the summary step writes no summary file at all when there are no URLs, and the script exits `3` because it cannot read one. So, from the exit code alone, "reached absolutely nothing" is distinguishable from "reached one thing". "Reached one thing" is not distinguishable from "scanned the whole application". ## Your own verdicts shrink what 0 can mean The buckets that the ladder reads are the ones left after several filters have already run: - a small built-in list of example and internal rule ids is dropped; - out-of-scope lines in the rule file discard findings by rule and URL; - findings the scan marked as informational risk are discarded unconditionally; - every rule you moved to `IGNORE` or `INFO` is kept but routed away from the gating buckets. Each of those is a legitimate choice. Together they mean that `0` is a statement about the findings you agreed to be gated on, not about the findings the scan produced. ## What to read instead | signal | what it tells you | |---|---| | the printed URL count | whether the crawl got past the entry point at all | | the per-bucket summary line | how many distinct rules landed in each bucket, including the ignored and informational ones | | the passing count | how many rules had no complaint -- useful as a trend, misleading as an absolute | | a report artifact from the same run | which URLs were actually visited | A practical rule for a pipeline: treat the exit code as the gate and the URL count as the **health check on the gate**. A run whose URL count collapses between builds has stopped testing your application, and its exit code will not tell you. ## How to make 0 mean more - Assert a floor on the URL count, or on the number of rules that ran, as a separate step -- a scan that suddenly sees one URL should fail loudly rather than pass quietly. - Keep the authenticated path working and check it, since a session that stops working is the single most common way coverage silently collapses. - Compare the per-bucket counts across runs rather than reading each run in isolation; the shape of the counts moves long before the exit code does. - Keep the rule file honest, so the set of rules routed away from gating is small and deliberate. None of that changes the exit contract. It changes what a green build is evidence of, which is the part the contract was never going to give you.