Your custom Rego rule for Trivy or KICS reports nothing — how do you diagnose it?

## The failure mode A custom rule that reports nothing and a custom rule that is working perfectly produce byte-identical output. That is what makes this the characteristic bug of Rego-backed scanner extension: there is no error, no warning, and a green pipeline. Somebody discovers months later that the control everyone believed was enforced never ran. Diagnose it in three layers, outermost first. ## Layer 1 — was the rule loaded at all? Extending Trivy, KICS or Terrascan means dropping Rego files somewhere and telling the scanner where. Each has its own custom-rule path option, and each has its own expectation about where the rule's `package` declaration must sit — a namespace the scan includes, distinct from the namespaces the built-in catalog occupies. Two ordinary mistakes live here: the CI job runs the scanner from a working directory where the relative path to your rules resolves to nothing, and the rule compiles fine but sits in a package the run does not evaluate. The check for this layer is not to read the rule again. It is to make the engine tell you what it loaded — run with the scanner's verbose/debug output, or deliberately break the rule's syntax and confirm the run now errors. A rule that produces no error when you corrupt it was never being read. ## Layer 2 — is the metadata there and correct? A Rego rule on its own is anonymous logic. Every one of these scanners requires identity to sit alongside it, because a finding without an id, a severity and a description is not usable by anything downstream. - **KICS** expects a query directory holding `query.rego` and a `metadata.json` beside it. The metadata carries the query's id, its severity, its category and its description text, and the platform the query applies to. The Rego itself contributes results into the `CxPolicy` result set with the fields the engine reports. - **Trivy** expects the rule to be annotated with a `# METADATA` comment block above the rule. Its `custom` section supplies the identifiers and the severity, and an input selector declares which kind of input the rule applies to, so a Terraform rule is not run against a Dockerfile. - **Terrascan** expects a Rego file paired with a JSON metadata file naming the rule, the resource type it applies to and its severity. Get the shape of that metadata wrong and the outcome ranges from "the rule is skipped" to "the rule runs but is reported at a severity nobody looks at" — which, from the perspective of a team scanning their dashboard, is the same as silence. This is also where the severity a finding carries is decided: it is a property the rule *declares*, and it is declared here, not inferred from the logic. One related trap belongs to this layer: if your custom rule declares an id that a built-in rule already uses, the id stops identifying anything. Reports show two different rules under one id, and any mechanism that names the id to silence a rule now silences both. Namespace custom ids to your organisation from day one. ## Layer 3 — is the Rego body undefined? This is the one that catches experienced people, because it is a language property rather than a tooling one. In Rego a rule body that references a key which does not exist in the input document does not evaluate to `false` — it is **undefined**, and an undefined body simply produces no result. There is no error and no violation. So a rule that walks `input.resource.aws_instance[name].instance_type` against a document whose actual shape is different silently matches nothing. The canonical mistakes at this layer are: assuming the document shape (each of these engines hands your rule its own parsed representation, and they differ from each other and from raw plan JSON), a typo in a key name, and a helper rule that is itself undefined so everything depending on it is too. The check for this layer is to evaluate the rule directly against a saved fixture of the exact document the engine passes in, and to build the rule up incrementally: first make the rule produce a result unconditionally, confirm it appears in the report, then add each condition and watch which one silences it. Whichever condition makes the finding disappear is the one whose path assumption is wrong. ## Making silence impossible next time The structural fix is to treat custom rules like code. Each rule ships with fixtures — a document that must violate it and a document that must not — and those fixtures run in CI. A rule that stops producing a violation on its violating fixture fails the build, which converts a silent nothing into a loud something. Without that, the only feedback signal you have is a report that looks the same whether the control is enforced or absent.