When a deployed target lacks a feature a case exercises, why should the case skip there rather than fail?

## A failure is a claim about the product Every outcome an automated run produces is a statement. **Failed** says the product did not do what was expected. **Passed** says the expected behaviour was observed. When a case is executed against a deployment where the feature it exercises simply is not present — the payment path is switched off, the outbound-message integration was never configured, the scheduled job does not run on a preview stack — neither statement is true, because nothing was learned about the product at all. Reporting that as failed is the more damaging of the two mistakes. A failure list that contains entries nobody intends to fix stops being read. Within a few weeks the team knows that "those eight are always red here", and the ninth entry — a genuine defect — joins them unnoticed. A run whose red is partly furniture has lost the one property that made it worth having. **Skipped** is the honest third statement: this check did not apply to this deployment. It is countable, it can carry a reason, and it does not consume anyone's attention as though something were broken. ## Declared capabilities beat probing at run time | | Declared capability | Probing at run time | |---|---|---| | Where the fact lives | an attribute of the resolved target | inferred from the product's own behaviour | | Reviewable | yes — a diff shows one appearing or disappearing | no — nobody sees the inference | | Failure mode | a wrong list blocks cases visibly | a wrong probe silently removes or admits cases | | Cost | one line per deployment | extra work at the start of every run | | Half-configured feature | still declared, so the case runs and fails honestly | often reads as absent, so the case quietly disappears | The mechanism has three parts, and none of them is inside a case body: 1. Each deployment's entry in the target table declares the **capabilities** it carries. 2. Each case declares the capabilities it **requires**, in one line beside the case itself. 3. Before executing a case, the runner compares the two. Where a required capability is not declared on the resolved target, the case is reported skipped and the missing capability is named on the result. Keeping the comparison in the runner matters for the same reason it matters elsewhere: a precondition each author has to remember to write is a precondition that exists only as often as it is remembered. ## Skips are a budget, not a free action A skip is honest, but it is not free — it is coverage that did not happen. A suite can drift until the deployment closest to release runs two thirds of its cases while everyone believes it runs all of them, and the report looks perfectly healthy the whole way. Two habits stop that: - **Every skip names its reason.** A skip that carries the missing capability aggregates into a single readable fact: this deployment skipped forty cases, all for one absent capability. A skip with no reason is noise, and noise is never reviewed. - **Every deployment declares what it is required to carry.** The run then fails once, on the gap between required and declared, naming the deployment — rather than shrugging forty times in a row. ## When a skip is the wrong answer - **The capability should be present.** If a deployment is meant to carry a feature and does not, the absence is itself the defect. Skipping the affected cases converts a misconfiguration into silence. - **The case is broken, not inapplicable.** A skip used to mute an unstable or failing case is a lie wearing the right word, and it is far harder to spot than a quarantine, because legitimate skips surround it. - **The deployment is the one that matters.** A check that skips everywhere except a developer's machine effectively does not exist. Track which capabilities are exercised on the deployment closest to release. - **The flag is a proxy for something else.** "Skip if slow", "skip on the shared deployment" and similar are not capabilities; they are unresolved problems being renamed. A capability describes what a deployment *has*, never how convenient it is. ## Practices 1. Keep the capability vocabulary small and describe features, not deployments. A capability named after a deployment is a conditional in disguise. 2. Make the required-capability declaration cheap enough that authors reach for it instead of writing their own guard. 3. Report skip counts per deployment with reasons attached, and treat a rising count as a change with an owner rather than as background weather. 4. Review the declared list when a feature is switched off anywhere; the list is the record of what each deployment is, and it is only as true as its last edit.