How do you set the precision-versus-recall operating point for an org-wide SAST program?

### The decision, stated properly Every security analyzer has a dial. Turn it toward **recall** and it reports more of the real defects, along with far more that are not defects. Turn it toward **precision** and nearly every report is genuine, at the price of quietly missing whole families. There is no setting that is correct in the abstract, because the constraint is not the analyzer - it is **the finite attention of the people who read the output**. Choosing the operating point is a capacity decision, and it is the part of a security-analysis program that a lead actually owns. ### Start from the reader, not from the rules Ask who is meant to act on each finding, on what clock, and what they are allowed to do about it. A developer looking at their own change in review has minutes and no security context; they can act only on findings that are almost certainly real and almost certainly theirs. A security reviewer working a queue has hours and full context; they can profitably sift a noisy stream. Those are two different audiences with two different tolerable precisions, and a program that sends one stream to both fails whichever audience it did not design for. That gives the standard two-tier shape: a **small, high-precision blocking set** whose findings are expected to be true and are fixed as part of the change, and a **broad, lower-precision advisory set** that lands in a review queue with an owner, an SLA and a triage budget. The interesting design work is deciding which rules qualify for the first tier, and the answer must come from measurement rather than from the vendor's confidence label. ### Credibility is a finite, non-renewable resource The failure mode that kills these programs is not a missed defect; it is a team that has learned the tool cries wolf. Once developers form the belief that findings are noise, they stop reading them, and recall on the blocking tier effectively goes to zero no matter what the analyzer reports. This is why precision on the tier that interrupts people is worth more than raw coverage, and why moving a rule *out* of the blocking tier is a legitimate, often correct action rather than an admission of defeat. ### Measure the program, not the findings count Total findings is a vanity number - it moves with codebase size and rule-set churn. Useful measures: * **Per-rule true-positive rate**, from a sampled triage of a fixed number of findings per rule. Rules below an agreed floor leave the blocking tier automatically. * **Time to triage and time to fix**, split by tier. A queue whose age is growing is a program that has already exceeded its capacity. * **Escaped-defect review.** When a security defect is found by any other means - a review, a penetration test, a report from outside - ask whether the analyzer could have found it and why it did not. That is the only honest estimate of recall you will ever get, because the false negatives are by definition invisible to the tool. * **Analyzer health.** Rules that failed to compile, files skipped, functions that hit the timeout, the share of the tree actually analysed. A green run that analysed a third of the code is worse than a red one, because it manufactures confidence. That last point is not theoretical. On a ride-hailing dispatcher's estate, the incremental-analysis cache was keyed on file modification timestamps, and a clock-skew artefact between two build agents - about 90 seconds apart - meant freshly changed files sometimes looked older than the cached result and were skipped. For 19 days the scan reported a healthy 0 new findings on a module that was being rewritten. Nothing in the findings dashboard could have shown that; only a coverage measure could. ### Make the accepted false negatives explicit A mature program writes down what it has decided *not* to catch this way - authorisation and business-logic defects, anything reached through dynamic wiring, whole languages or repositories with no usable rule set - and names the compensating control for each: design review, a targeted manual assessment, runtime testing, a monitoring detection. An undocumented gap becomes, over time, a belief that the gap does not exist. ### Fund the triage, and place it where the knowledge is Precision is also an economic choice about somebody's week. An 11-person team cannot absorb a thousand-item advisory queue on top of delivery, and pretending otherwise produces a queue that is formally owned and actually abandoned. Either the volume is cut to the capacity, or capacity is added - a rotating triage duty, a security partner, a budget for model work. And the model work is the compounding investment: every false-positive family closed by declaring a team validator or narrowing a sink rule raises precision permanently, for every future scan, which is what lets you move the dial back toward recall without spending more attention. ### The framing to leave the interviewer with Set the operating point per audience; buy precision on the interrupting tier with rules you have measured; keep recall on a tier somebody is genuinely funded to read; measure the analyzer's own coverage as carefully as its findings; and write down what you have chosen not to catch.