A custom vet analyzer every repo runs has reported nothing for a quarter. How do you prove it still fires?

## Silence is not evidence A false negative in a static check is uniquely hard to notice: the build is green, nobody files a ticket, and everyone assumes the rule is being enforced. The only defence is to make the check prove itself continuously, and the only way to debug one is to take the chain apart layer by layer, because at least five different layers can swallow a diagnostic. ## Layer 1 — does the matching logic fire on a fixture? `golang.org/x/tools/go/analysis/analysistest` exists for this. `analysistest.TestData()` returns the calling package's `testdata` directory; `analysistest.Run(t, dir, Analyzer, "a")` analyses the package `a` beneath it and compares the diagnostics against `// want` comments in the source. Each `// want "regexp"` asserts that a diagnostic whose message matches that regular expression is reported on that line; an unmatched expectation and an unexpected diagnostic both fail the test. The discipline that actually prevents the quarter-long outage is: **for every rule the check enforces, keep both a positive fixture and a near-miss negative fixture.** The positive one fails loudly if the matching logic regresses. The negative one — code that looks similar and must *not* be flagged — fails if somebody widens the check to make the positive pass again. `analysistest.RunWithSuggestedFixes` extends this to fixes, comparing the rewritten source against `.golden` files in the same tree. If the fixture does not report, the bug is in the check itself: most often a syntax-shaped match that the real call sites do not have, or a type predicate narrowed to one spelling of an import. ## Layer 2 — is the analyzer in the binary anyone runs? A suite built with `multichecker` only contains the analyzers passed to `Main`. A check added on a branch and never added to that list passes its own tests forever. Run the shipped binary directly over a package with a planted violation. If the fixture reports and the binary does not, the analyzer is either missing from the driver or disabled by its own flag — drivers expose one flag per check, and a check switched off during a rollout tends to stay off. ## Layer 3 — are the packages being analysed at all? `go vet` analyses the packages matching the patterns you gave it. A repository whose command names a subset, a directory excluded by the pattern, or generated code the team routinely skips all produce genuine silence. Related and easier to miss: files excluded by build constraints or by `GOOS`/`GOARCH` never appear in `pass.Files` at all — they land in `pass.IgnoredFiles`. A rule about platform-specific code checked only on one platform's build is silently unenforced on the others. ## Layer 4 — was the analyzer skipped? If a package does not type-check, the driver does not run analyzers on it unless the analyzer sets `RunDespiteErrors`. In a healthy repository this rarely matters, because the build would fail too — but in a repository with a package that is only ever built under a tag, or with vendored code that no longer type-checks, whole subtrees can be quietly skipped. ## Layer 5 — is the result cached? The go command caches vet results, keyed in part on the identity the tool reports for `-V=full`. A rebuilt analyzer that does not change that identity can therefore serve results computed by the previous version. The first time a change to a check appears to do nothing on unchanged packages, this is worth ruling out — touch the input or clear the build cache and compare. ## Turning the answer into a control The diagnosis is the easy half; the reason it lasted a quarter is that nothing was watching. Two cheap controls fix that: 1. **Fixtures in the check's own repository**, run in its tests, covering every rule with a positive and a negative case. This is the golden-file discipline: a regenerated expectation is reviewed as a diff, so widening a check is a visible change rather than a silent one. 2. **A canary in the consuming build** — a tiny package, excluded from the shipped binary, containing one deliberate violation, where the *expected* outcome is a diagnostic. If the check ever stops running end to end, the canary stops reporting and that is observable. And when the check is later found to have missed real code, the fix is not only to widen the matcher: it is to add the missed shape as a fixture, so the same gap cannot reopen. ## The habit worth stating in an interview Every new check should be introduced by first watching it fire. Run it over the whole corpus in a non-blocking mode and read a sample of the hits and, more importantly, a sample of the places you *expected* a hit and did not get one. A check that reports zero findings on its first run over a large codebase is almost never a clean codebase; it is almost always a broken check.