What does opa test --coverage mark as covered in a Rego policy, and what does it not prove?

## What the flag actually records Running `opa test --coverage` turns on an evaluation tracer: while the test run executes, OPA records the source location of every expression the evaluator reaches. The report (readable as JSON with `--format=json`) is organised per file — a list of covered source ranges, a list of not-covered ranges, counts of each, and a coverage percentage for the file — plus an overall percentage across the files that were loaded. The unit is a source location, not a "line" in the sense a Go or Java coverage tool means. A Rego rule body is a conjunction of expressions, and each expression is tracked on its own. That granularity matters, because in Rego the interesting failures live inside a body rather than between statements. ## What "covered" means, precisely Covered means one thing only: during this run, the evaluator reached that expression at least once. It does **not** mean the expression evaluated to true. It does **not** mean a test asserted anything about the result of the rule containing it — the coverage tracer has no idea which assertions exist or which of them passed. And it does not mean the enclosing rule produced a decision. ## The undefined interaction, where Rego differs from most languages A rule body is a conjunction, and evaluation stops at the first expression that is **undefined**. Undefined is not `false`; it is the absence of a value, and a body that goes undefined produces no result at all rather than a negative one. So coverage within a body is prefix-shaped: if a fixture makes the first comparison undefined, nothing below it is covered. Read positively, that is the report's most actionable signal. An uncovered tail on a `deny` rule means no test in the suite ever drove that rule all the way to its message — nothing you have written has ever seen the rule deny. A percentage averages that away; the `not_covered` ranges name it. ## What 100% does not buy you Take a rule requiring that every namespace has an entry in a quota inventory loaded as `data`. Suppose every expression in it is covered. All that establishes is that at least one fixture reached the end of the body: one namespace object, one inventory, one message. Coverage is over the policy **text**. What decides whether the guardrail works is the **input space**: - the namespace whose name matches a record with different capitalisation; - the object where the metadata your rule reads is simply absent; - the inventory document that was never loaded at all — in which case the reference into `data` is undefined, the rule quietly produces nothing, and the outcome is indistinguishable from "allowed". None of those appear as coverage gaps, because the same expressions are the ones being evaluated. A rule that denies everything can report exactly the same coverage as a correct one. So can a rule whose message is wrong, or a rule that is never reached in the environment where it is deployed. ## Reading it as a lead, before a rollout The honest framing at sign-off: coverage is a floor detector. It finds untested policy cheaply and reliably, and that is worth having. It is not evidence that the rule is right. So do not sign off on the number. Ask for: - at least one test per `deny` message that makes the message appear, and one that proves the rule stays quiet on a legitimate object — a rule that only ever fires is as broken as one that never does; - fixtures loaded with a document shaped like the real reference data, not a two-record stub, so the `data` lookups in the body are exercised against a realistic shape; - a walk of the `not_covered` list looking for whole rules and helper functions that no test touches — those are either dead policy or the branch that will surprise you; - an explicit case for the "reference data missing" scenario, because that is the failure that turns a deny rule into a no-op. ## Using the number honestly Treat rising coverage as "we have stopped shipping untested rules" and nothing more. The claim "the guardrail works" is carried by which decisions the tests pin down, not by how much of the file was executed while pinning them.