The Life of a Rule

Near-miss cases are legitimate inputs sitting just inside the boundary — the changes the rule must let through. Deny cases only prove a rule can refuse something. They never prove it refuses nothing else.

The input contract is the written agreement about the document a rule decides over: which step produces it, how it is wrapped, and which fields are guaranteed present. Agree it before writing the rule.

Which ruleset produced the number and what it covered. Zero violations only means the rules that actually loaded found nothing in the resources that were actually enumerated. An empty or replaced ruleset reports exactly the same green.

A rule is production code: one bad rule blocks every team's builds at once. Review, tests and CI catch it before it reaches a gate, and give each change an author, a reviewer and a history.

A rule's id is the handle everything outside the rule keys on: waivers, suppressions, control maps, dashboards. Titles get reworded, so they cannot be that handle. The owner tells a blocked engineer who to ask.

A synchronous gate runs inside someone else's request, so its time is added to theirs. The p99 target bounds the tail that callers actually feel, and the hard timeout caps the worst case so the caller's own deadline stays predictable.

Capture the exact input document the engine evaluated and the version of the rules it used, then replay that pair on your own machine. Reproduce the decision before you start editing the job definition.

Not yet. Zero denials is ambiguous: it can mean nothing in scope violated the rule, or that the rule never matched anything at all. Compare how often it was evaluated with how often it denied before deciding.

The same fact has several representations in the document. A secret reaches the environment through env[].valueFrom.secretKeyRef and through envFrom[].secretRef, and the rule inspected only the first. The contract must enumerate every field the fact can appear in.

Anything that moves the shape the rule reads: a field renamed or relocated between API versions, a kind served under a new group-version, a field that gains a default so it is never absent, or an engine change to how the input document is built.

Separate an engine failure from a working rule. The platform on-call owns the engine; the named rule owner owns the decision. The failure output must name the rule, the resource and property, the owning team and how to propose a change — otherwise every block routes to the platform team.

Confirm it is version drift rather than a false positive by running both versions on the same manifest and reading the changelog between them. Then unblock by fixing the manifest, and close the window by moving the lagging hooks forward. The gate stays current.

Each decision point installs its own copy of the rule library and updates on its own schedule. Publishing a new version does not change what is already installed, so the three run different rule versions until each one is upgraded.

A rule that stops matching anything denies nothing, and that silence looks exactly like success. A fixture the rule must deny turns the silence into a failing test: if it suddenly passes, enforcement has quietly stopped.

Triage all 118 denials first: a denial is a false positive only if the image actually complied. The rate is confirmed false positives over the images evaluated. A raw denial count is not a rate.

A hand-written fixture encodes the author's mental picture of the input document — the same picture the rule encodes. If that picture is wrong, both are wrong together and the test still passes. Captured documents carry shapes nobody imagined.

By computing a digest over the whole rule set it loaded and comparing it against an expected value obtained through a different path than the ruleset itself. A mismatch must fail the run loudly, and the digest should be stamped on every result.

It must assert what the rule allows, not only what it denies — otherwise a rule that blocks everything passes. It also needs the awkward cases: the property missing entirely, and a value that is only resolved at deploy time.

Collapse near-copies when the logic is identical and only a data value differs - the list of allowed licences, say. Keep them separate when the denial message or the remediation genuinely differ, because that is different logic wearing the same shape.

Anything that turns a previously passing input into a failure: a new blocking rule, a tightened condition, or a stricter default. Changes that cannot newly fail anything — a fix that only loosens a check, a rule shipped switched off — are minor or patch.

Mostly in two terms that multiply: marshalling the whole inventory once per call, then every rule walking every listener. Cost tracks rule count times input size, so one extra rule is paid on every call and on every item.

One of the decision's inputs differs. Either you replayed a rebuilt document rather than the recorded one, a different rule revision, or reference data that has since changed. Chase the difference; do not call the gate flaky.

Because the corpus only holds changes people actually made and completed within that window. Rare events like a quarterly base refresh or an incident rebuild fall outside it, and a zero can also mean the rule never fired at all.

A verdict-only assertion passes when the rule denies for the wrong reason, so a broken condition stays hidden behind a green test. The message is also the whole product for the developer who is blocked, so it deserves an assertion of its own.

Because the fixtures were hand-written from documentation while the enforcer sends a defaulted, already-rewritten document. The tests prove the rule works on a shape nobody produces. Validate captured real output against the declared contract in CI.

Not from the results — they look perfect. Detect it out of band: alert on every publish to the rule distribution point, reconcile the enforcer's reported ruleset digest against what was actually published, and keep a deliberately non-compliant canary whose finding must appear in every sweep.

Everything keyed on the old id stopped matching, silently. Waivers now exempt nothing, suppressions in other repositories are dead text, control-map rows name a check nobody emits, and the rule's violation trend fell to zero - which reads like success.

The cache key covered the object's identity, not the content the rule reads. Nothing in the name or port changed when the TLS minimum did, so the gate returned an old allow without evaluating anything.

Diff the documents the engine actually evaluated against the shape the rule author assumed. If expansion produced entries where the field the rule reads is absent rather than wrong, the rule met an input it was never written for.