Engineers ignore your analyzer because undecidability forces it to approximate, so which structural changes buy back precision, and at what cost?
answer
- precision is bought, not configured
- change the question, not the tool
- restrict, annotate, bound, or re-aim
- each escape spends freedom or effort
- authority should match error direction
basics
~20 sFour escapes exist, and each changes the question rather than the tool: restrict what may be written, have authors assert facts the tool cannot derive, bound the search, or replace the behavioural rule with a structural proxy. Each is paid for in expressiveness, effort or coverage.
solid answer
~50 sTuning cannot help, because the limit lives in the question rather than in the implementation. Every practical escape changes the question. You can **restrict the language** a component may use, so the construct that defeats the analysis cannot appear, and pay in expressiveness and migration. You can **ask the author** for annotations or contracts that supply the fact the tool cannot derive, and pay in ongoing effort plus a boundary with unannotated code. You can **bound the exploration** to a fixed depth or step budget, buying exactness inside the bound and silence beyond it. Or you can **re-aim at the text**, replacing the behavioural rule with a structural one that is exact but catches only what matches the shape. The process layer matters as much: let only proof-carrying checks block a merge, and give the rest advisory status.
go deeper
Notice that an annotation you write is doing work the tool could not do by itself, which is why it is being asked for rather than inferred.
Be able to name the escapes — a narrowed language, author-supplied facts, a bounded search, a structural proxy — and say what each one gives up in exchange.
Show how you would roll one out: which surface it covers, how the boundary with uncovered code is handled, and what you measure to know whether it is working.
Make the trade explicit as a budget. Each escape spends engineer freedom, engineer time or coverage, so scope it to the code whose failures are genuinely expensive and refuse the uniform version.
## Why tuning does not help The first instinct when a gate is ignored is to tune it: more analysis time, more context, a better model. Precision does improve that way, and it is worth doing, but it has a ceiling that no budget passes. Whether a program's behaviour has a given nontrivial property has no exact procedure at all, so every version of the tool is approximating, and every version has the same two failure modes available to it. If the team's complaint is 'it is sometimes wrong', no amount of tuning answers the complaint. What answers it is changing what the tool is asked. ## Four escapes, each with a bill 1. **Restrict what may be written.** Define a subset for the component that matters: no dispatch to targets the analysis cannot enumerate, no code loaded at runtime, no unbounded loops in the section under proof. Inside that subset the question can become genuinely decidable, and the tool becomes exact. The bill is expressiveness and a migration, plus the long-term cost of a second dialect that new joiners must learn and that library code will not respect. 2. **Ask the author.** Annotations, contracts, invariants and ownership markings supply the fact the tool could not derive. The analysis stops guessing and starts checking consistency, which is a local, decidable job. The bill is per-site effort, drift as code changes, and a boundary problem: at the edge with unannotated code you are back where you started, and the honest design decides in advance whether the tool *verifies* an annotation or merely *believes* it. 3. **Bound the exploration.** Analyse up to a fixed number of loop unrollings, call depth or steps. Inside the bound the answer is exact and the findings are real. The bill is silence beyond the bound, and the temptation to report that silence as a proof of absence, which it is not. 4. **Change the property.** Replace the behavioural rule with a structural proxy that is exact and conservative — a rule about the shape of the code rather than about what it does. The bill is that the proxy rejects safe code that happens to match the shape, and misses unsafe code that does not. ## Matching a check's authority to its error direction | Check's error direction | What a finding means | Appropriate authority | |---|---|---| | Only speaks with a proof | the finding is real by construction | may block a merge | | Speaks when it cannot rule out | the finding is a hypothesis | advisory, with a review path | | Bounded search, exhibits an input | real, but coverage is limited | may block, but proves nothing on silence | Most failed rollouts get this wrong in one specific way: they put hypotheses behind a blocking gate. The team cannot ship until it disposes of each one, disposal is faster than investigation, and within a quarter the suppression list is the real policy and nobody reads it. The direction of a check should decide its authority, and that decision belongs in the check's own definition rather than in a global severity setting. ## The organisational read Every one of the four escapes spends something an engineer owns: their freedom to write what they like, their time writing assertions, or the coverage they thought they were getting. That makes this a language-and-process decision rather than a tooling purchase, and it is why it lands with a lead. The useful framing in a planning discussion is: *which of these are we willing to pay, on which code?* Almost nobody can pay the language restriction across a whole codebase, and almost everybody can pay it on the small surface that handles money, credentials or data loss. The same is true of annotations. Scoping the escape to the code whose failures actually hurt is the move; applying it uniformly is what makes it fail. ## What to measure - The **survival rate** of each check's findings — what fraction lead to a change. A check whose findings never survive is not a precision problem, it is a check to retire. - The **size and age of the suppression list**, per check. Growth is the leading indicator that a hypothesis is wearing a proof's authority. - The **annotated fraction** of the surfaces you claim to have covered, because a partially annotated boundary gives a guarantee that does not hold. - The **escape-hatch usage** in the restricted subset, which tells you whether the restriction is real or ceremonial. ## The claim to never make Do not promise exactness at the end of the roadmap. You can promise a narrower language on the critical surface, findings that carry proofs, a bounded search that exhibits real inputs, and a suppression path with a name attached. The exact answer to a nontrivial behavioural question is not a thing any budget buys, and a plan that implies otherwise loses credibility on the first counterexample an engineer produces.
- Which findings should block a merge, and which should only advise?Block on checks whose error direction is under-reporting, where the tool speaks only when it has a proof or an exhibited input, because such findings are real by construction. Advise on over-approximating checks, whose findings are hypotheses. Putting both behind one blocking gate is what teaches a team to suppress everything it sees.
- What is the characteristic failure mode of buying precision with annotations?Annotations are promises, and promises drift. If the tool trusts what it cannot check, the analysis is only as sound as the least careful author. If it does check them, the burden lands on every site and adoption stalls at the boundary with unannotated code. Decide which of the two you are doing, and budget the boundary explicitly.
- A team proposes restricting the language across the whole codebase to make analysis exact. How do you respond?Scope it. A restriction pays for itself on the surface whose failures are expensive and is rejected everywhere else, because the expressiveness cost is paid daily by everyone while the benefit concentrates in a few paths. Ask which code the guarantee is actually needed on, and apply the subset there with a reviewed escape hatch.
saying these in an interview costs you the question
- Promises exactness once the analysis is tuned enough
- Blocks merges on checks that report unproven hypotheses
- Adds annotations without deciding whether the tool verifies them
- Treats a growing suppression list as a configuration detail
- Reads a bounded search's silence as proof of absence
- Applies a language restriction uniformly across an entire codebase