Why does a Rego test whose only assertion is deny with input as bad_pod always pass?
answer
- only false is falsy in Rego
- a partial set rule is always defined
- empty set is still a value
- truthiness is not membership
- assert count, member, or exact text
basics
~20 sA partial set rule such as deny always produces a document, empty when nothing matched, and in Rego every value except false is truthy. So the expression succeeds whether or not the rule fired, and the test asserts nothing.
solid answer
~40 s`deny` built as `deny contains msg if { ... }` is a partial set rule. Such a rule is always defined: if no binding satisfies the body, its value is the empty set rather than undefined. In Rego an expression fails only when it is undefined or literally `false`, and an empty set is neither, so `deny with input as bad_pod` succeeds vacuously — the test is green even after the rule stops firing entirely. The mirror mistake is asserting an allowed case with `not deny`, which can never succeed for the same reason. Assert on the contents instead: membership, `deny[expected_msg]`; cardinality, `count(deny) == 1`; or strongest, set equality against the exact message, which also catches a rule that fired for the wrong reason with different text.
code
rego · 9 lines# always green: an empty set is defined and not false
test_missing_probe_denied_vacuous if {
deny with input as no_probe
}
# fails if the rule stops firing, fires twice, or changes its wording
test_missing_probe_denied_pinned if {
deny == {"payments-api: no readinessProbe on container"} with input as no_probe
}go deeper
Remember that in Rego only false is false: an empty set, an empty string and zero are all truthy, so naming a set in a test proves nothing about its contents.
Explain why a partial set rule is defined even when it collects nothing, and rewrite a bare assertion into one that pins the count and the exact message.
Diagnose a green suite over a rule that no longer fires, and show how you verify a contributed failing test is red for the reason its author thinks it is.
Decide what the review bar is for a rule change — which assertions a rule may not merge without — and who arbitrates when a blocked team's test and your rule disagree.
## Undefined, false, and the thing in between Rego expressions have three outcomes: a value, `false`, or **undefined**. A rule body fails only on the last two. Crucially, *every* value other than `false` is truthy — including `0`, `""`, `[]` and `set()`. Now ask what kind of document `deny` is. A gate rule is normally a partial set: ```rego deny contains msg if { input.kind == "Deployment" not has_readiness_probe(input) msg := sprintf("%s: no readinessProbe on container", [input.metadata.name]) } ``` (The older syntax `deny[msg] { ... }` you will still meet in existing policies means the same thing.) A partial set rule collects every `msg` for which the body holds. If the body holds for nothing, the result is the **empty set** — a perfectly good, defined value. It is not undefined. Put those two facts together and the trap is complete. `deny with input as bad_pod` is an expression that evaluates the set and asks only whether it is defined and not false. It is always defined. It is never false. The test passes. ## What that costs in practice The failure mode is not that the test is weak, it is that the test is *inert*. Somebody refactors the rule and mistypes the container path, or narrows the `input.kind` check, or moves the rule under a helper that no longer matches. The rule now denies nothing at all. The gate silently stops blocking. And the suite is still green, because the assertion never depended on the rule firing. This is worse than having no test, because the green suite is what everyone points at during review. ## The mirror mistake The same fact bites in the opposite direction. Asked to assert that a compliant manifest is allowed, the natural-looking line is: ```rego test_compliant_is_allowed if { not deny with input as good_pod # always fails } ``` `not <expr>` succeeds when the expression is undefined or false. `deny` is defined and truthy even when empty, so `not deny` never succeeds and this test can only fail. People then "fix" it by deleting the assertion rather than by understanding it. ## Assertions that actually hold the rule down In rough order of strength: - **Cardinality** — `count(deny) == 1 with input as bad_pod`, and `count(deny) == 0 with input as good_pod` for the allow case. This is the minimum bar, and it fixes both traps above. It also catches the common bug where one violation produces two messages because a loop iterates twice. - **Membership** — `deny[expected_msg] with input as bad_pod`. A reference into a set with a concrete key is undefined when the element is absent, so this genuinely fails when the message is missing or its text drifted. - **Exact set equality** — `deny == {"payments-api: no readinessProbe on container"} with input as bad_pod`. The strongest of the three: it pins both the count and the exact text, so a rule that fires for a different reason and produces a different message fails the test rather than quietly satisfying it. Pinning the message text is not pedantry. The message is the product: it is what the blocked developer reads at 5pm, and it is the only guidance most of them will ever get about the rule. A rule that denies the right thing while saying the wrong thing is a real defect, and only a text assertion catches it. The same discipline applies to the waived branch. "No message" is what a waiver is supposed to produce, so assert `count(deny) == 0` with the waiver document stubbed — never a bare `deny`, and never `not deny`. ## How this reads from the other chair When a developer whose change your rule blocked sends a pull request against the policy repository, the useful artefact is a test that is *red*: their manifest as a fixture, and an assertion about what should have happened. Verify that it is red for the right reason before you touch the rule — a contributor unfamiliar with Rego is quite likely to have written the vacuous form, and a test that cannot fail is not evidence of anything. Once the assertion pins the exact message, the argument becomes concrete: either the rule is too broad and the message text and count you agree on becomes the new pinned behaviour, or the rule is right and the same test flips to asserting the deny, permanently recording why.
- Why does asserting not deny fail to test the allowed case?`not` succeeds only when its expression is undefined or false. A partial set rule is defined even with no members, and an empty set is not false, so `not deny` never succeeds and such a test can only ever fail. Use `count(deny) == 0` with the fixture attached instead.
- You inherit a suite full of bare deny assertions. How do you find out whether the rules still work?Make each rule fail on purpose and confirm the suite goes red — comment out the predicate, or point the fixture at a compliant object. Any test that stays green is asserting nothing. Rewrite those to pin count and message text before touching the rules themselves, so later changes are actually held down.
- Is asserting the exact message text too brittle?It breaks when the wording changes, which is the point: the message is the only guidance the blocked developer gets, so a deliberate reword deserves a deliberate test update. If several tests share one string, hoist it into a rule they all reference so a wording change is one edit rather than twenty.
It is like asserting that a bug tracker exists rather than that it contains the bug you filed. The query succeeds either way.
saying these in an interview costs you the question
- Says an empty set is falsy or undefined in Rego
- Uses not deny to assert that an input is allowed
- Treats a green suite as proof the rule still fires
- Asserts only the count and never the message text