Why can an editor report a file clean while the build fails the same analysis rule on it?

### "It's clean here" is a claim about a specific run When an editor reports a file clean and the pipeline fails the same rule on the same file, the instinct is to call it flaky. It almost never is. Static analysis is deterministic: the same analyser version, over the same bytes, with the same effective configuration, produces the same findings. So the disagreement is evidence that one of those three inputs differs, and the diagnosis is a matter of finding out which. ### The usual causes, roughly in order of frequency **1. Different analyser versions.** Editor integrations often ship with a bundled analyser and use it unless told otherwise, while the pipeline uses the version pinned in the project. Rules are added, renamed, split and have their defaults changed between releases, so a newer engine in the pipeline reports things an older bundled engine never had. Fix: pin the version in the project manifest and configure the editor to use the project-local analyser. **2. A different config file was resolved.** Most analysers search upward from the file being analysed for the nearest configuration, and many merge a user-level or editor-level config on top of the project one. Open a file inside a nested module and the editor may resolve that module's config; the pipeline, invoked from the repository root, resolves the root's. Per-path overrides make this worse, because both runs then apply a real config — just not the same one. **3. Different file sets or different content.** The editor analyses the buffer, including unsaved edits; the hook analyses staged content; the pipeline analyses committed content on the merged result. Three different byte streams. Add ignore files honoured by one layer and not another, generated sources that only exist after a build step in the pipeline, and case-sensitivity or line-ending normalisation differences between a developer machine and a build machine, and you have several ways for "the same file" to differ. **4. Missing analysis inputs.** Rules that need type information, resolved dependencies or a compiled model degrade quietly when that context is unavailable — often to *fewer* findings rather than an error. An editor with a fully indexed project can therefore see a violation the pipeline misses, or the reverse, when the pipeline resolves dependencies the editor never fetched. This class is the one that produces the most confusing divergences, because both runs look successful. **5. Environment-dependent rule behaviour.** A ruleset keyed off an environment variable, a locale, a default encoding or an operating-system path convention will behave differently on a build machine than on a laptop. In a quote engine we once chased a rule about currency rounding that fired only in the pipeline: the rule was enabled by a profile the build activated and nobody's editor did. **6. Stale caches.** Analysers cache results keyed on file content and configuration. A cache key that does not include the configuration or the analyser version will happily serve findings computed under the old rules. ### How to diagnose it in minutes rather than hours Make both runs *say what they did*. Have the entry point print the analyser version, the absolute path of the resolved configuration and the number of files in scope, and print it in the build log as well as in the editor's output. Comparing three lines usually ends the investigation. If the printed values match, hash the effective configuration — many analysers can dump the fully merged, resolved ruleset — and compare the hashes; that catches a user-level override merging on top. Then reproduce deliberately: run the pipeline's exact command locally on the committed content, not on your working tree. If it fails locally, the difference was content or file set; if it passes locally, the difference was the environment or a build-produced input. ### The structural fix Diagnosis is cheap once, expensive every week. The durable fix is to remove the opportunity for drift: - One configuration file, committed, at the repository root, with per-path overrides expressed inside it rather than by scattering config files. - One pinned analyser version, resolved from the project, used by editor, hook and pipeline alike. - One entry point — a single task or script — that all three call, so the pipeline definition contains no analysis flags of its own. - Version-print in the log so drift is visible the first time it happens. Two responses mark a weak candidate here: calling it flaky, and "fixing" it by disabling the rule in the pipeline so the build goes green. The second converts a tooling inconsistency into a permanent hole in the gate.