Your policy suite is green but no captured plan ever exercised one rule branch — how do you close that gap?
answer
- capture samples the past only
- enumerate from the rule, not the corpus
- what if the attribute is absent
- no match means no violation means allow
- derive synthetic cases from real documents
basics
~20 sEnumerate the rule's branches by reading the rule, not the corpus, then hand-write a fixture for each branch reality has not yet produced. Captured plans only contain what teams have already done, so whole conditions can sit untested behind a green suite.
solid answer
~50 sA captured corpus is a sample of past merged work, so it is systematically blind to branches nobody has exercised. I close the gap from the rule side: list every condition the rule tests and, for each attribute it reads, ask what happens when that attribute is absent, when it is supplied by a shared module's default rather than written in the change, and when several resources appear and only one offends. Then I construct a fixture for each combination the corpus does not contain and mark it synthetic in the table so reviewers know it is not ground truth. The branch that most often turns up broken is the missing-attribute one: if the region field is not present in the resource, the condition never matches, the rule yields no violation, and the gate reads that as allow. A rule that says nothing is a rule that permits.
go deeper
Know that fixtures taken from past changes only cover situations that have actually happened, so some parts of a rule may never have been run by the suite.
Be able to enumerate a rule's branches from the rule itself — present, refused, absent, indirectly supplied, several resources — and say which of those a captured corpus is unlikely to contain.
Show the diagnosis: state correctly that a rule body which matches nothing yields no violation and therefore permits, and prioritise the branches whose failure mode is silent permission over the ones that fail loudly.
Own the rule for the whole estate: which branches must carry a case before a rule may be enforced, and how you keep a suite from drifting back to constructed evidence over time.
## Why a green suite can be silent about a branch Fixtures captured from merged changes give you realistic shapes and honest verdicts, but they are a sample of what teams have already done and had approved. If every team in the estate writes the region explicitly in the resource block, then no captured plan will ever contain a resource whose region arrives from a shared module's default or from the provider configuration. The rule's branch that handles that case has never been evaluated by anything. It is not weakly tested; it is untested, and the suite is green. That is the structural limit of capture, and the fix is not more capture. It is to work from the rule instead of the corpus. ## Enumerate branches from the rule, not the corpus Read the rule and write down, for each attribute it reads, the states that attribute can be in: - **present with a value the rule approves** — the allow path; - **present with a value the rule refuses** — the deny path; - **absent from the resource** — inherited from a module variable, a provider default or a workspace-level setting; - **present but produced indirectly** — the resource comes from a shared module, so the address and nesting differ from a directly declared one; - **present in several resources at once**, only one of them offending; - **present on a resource being updated rather than created**, if the rule is meant to judge creation only. For an instance-family-and-region rule, that grid is small enough to write out on one screen, and each cell is either covered by a captured plan or it is not. The cells with no captured plan behind them are the ones you construct by hand. ## The failure this usually uncovers The absent-attribute cell is where rules break most often, and the reason is a property of how these engines decide. A deny-style rule contributes a violation when its body matches. If the attribute the body reads is not there, the body simply does not match — it produces no result. No result is not a denial; the change passes. So the rule that was supposed to guarantee "every instance is in an approved region" quietly permits every instance that does not state a region at all, which is the exact case a module default produces. Writing that fixture — a resource with the region key absent — and asserting the outcome you actually want is what turns the branch from an assumption into a decision. Often the right answer is that a missing attribute is itself a violation, because the rule cannot vouch for a value it cannot see. Sometimes the right answer is to resolve the default first, before the rule ever runs, so that the document handed to the engine always states a region. Either way, the question has to be answered explicitly rather than inherited from whatever the rule happens to do when it matches nothing. ## Keeping synthetic cases honest A constructed fixture is weaker evidence than a captured one, because you are once again asserting your model of the document. Three habits limit the damage: - **Derive, do not invent.** Build the synthetic case by editing a captured plan — delete the region attribute from a real resource rather than typing a resource from scratch. The rest of the document stays realistic. - **Label it.** Mark the row as synthetic in the table, with a one-line note saying which branch it exists to reach. When a reviewer later asks why this shape is even possible, the note is the answer. - **Promote it when reality catches up.** The first time a real change produces that shape, swap the synthetic fixture for the captured one and keep the same expectations. ## Ordering the work Not every uncovered branch is worth a fixture today. Rank by what happens if the branch is wrong: a branch whose failure mode is *permit silently* outranks one whose failure mode is *deny loudly*, because the loud one will be reported within a day by the person it blocked and the silent one may never be reported at all. On a rule with a dozen branches that ordering usually puts the missing-attribute and indirect-declaration cases first and the many-resources case somewhere in the middle. ## What a strong answer sounds like Name the blind spot as a property of capture rather than a mistake; enumerate branches from the rule; identify the missing-attribute case and state correctly that an unmatched rule yields no result and therefore permits; construct the fixture by editing a real document; label provenance; and prioritise by failure direction. An answer that just says "add more test cases" has not shown how you would know which ones are missing.
- Why is the missing-attribute branch the one that usually turns out to be broken?Because a deny-style rule only contributes a violation when its body matches. If the attribute is not in the document, the body matches nothing, the rule returns no result, and the gate treats an empty result as permission. Authors read their rule as asserting a requirement, but it only ever objects to values it can see.
- Is the right fix always to deny when the attribute is absent?Not always. Denying is the safe default when the rule cannot vouch for what it cannot see. The alternative is to resolve defaults before evaluation so the document always carries a concrete value, which keeps the rule simple but moves the burden onto whatever produces the input. What is unacceptable is leaving the case undecided and discovering the answer in production.
- How do you stop synthetic fixtures from quietly outnumbering captured ones?Label provenance per row and look at the ratio when the rule is reviewed. A table that is mostly synthetic has drifted back to testing the author's model of the world. Promote a synthetic case to a captured one the first time a real change produces that shape, and challenge any branch that has stayed synthetic for a long time — it may not be a shape the estate can actually produce.
A road tested only by the cars that have driven it. Nothing proves the bridge holds a lorry until you drive one across on purpose.
saying these in an interview costs you the question
- Assumes a green captured suite means every branch ran
- Thinks an unmatched rule body denies by default
- Adds cases at random instead of enumerating conditions
- Types synthetic documents from scratch rather than editing real ones
- Never records which fixtures are constructed