A client wants the executive summary of an AI red-team report to read "no critical issues found", but the engagement's hours only allowed two of the eight planned attack categories to be exercised. How do you word the conclusion and the remainder section?
answer
- bound inside the sentence, not the appendix
- unbounded true sentence = certificate
- remainder = next engagement's scope
- quotes drop the qualifier
- negotiate placement, never presence
basics
~20 sKeep the true finding and bound it. Say no critical issues were found in the categories actually exercised, name them, and list the six that were not, with the reason. A summary saying only "no critical issues found" lets the reader treat untested areas as cleared, which is a claim the evidence cannot carry.
solid answer
~50 sThe client's sentence is not false, it is unbounded — and an unbounded true sentence is how a report becomes a safety certificate for work nobody did. Write the bound into the same sentence, not into an appendix: *"Within the two categories exercised (X and Y), no critical issues were identified. Six of the eight planned categories were not exercised; they are listed in section N with the reason."* That costs you the clean headline and buys three things. The reader can size the assurance. The people who allocated the hours see what those hours did not buy — which is usually the argument for a follow-up. And you are not on the hook when a later incident lands in an untested category. If the client pushes, negotiate placement and tone, never presence: retitle the remainder as agreed limitations, move the reasons alongside, offer a scoped follow-up. What you do not do is let a summary imply coverage the body does not support.
go deeper
Says the summary must state which categories were tested and that the rest were not.
Writes the bounded conclusion and a remainder list with reasons, and can explain why the unbounded sentence is misleading despite being true.
Handles the client pressure — offers placement and tone changes, frames the remainder as follow-up scope, and keeps the qualifier inside the quotable sentence.
Owns the firm's line on what a deliverable may claim, escalates rather than edits when a client insists, and makes the bounded-claim template the default so juniors are not negotiating it alone.
This is the leaf's central tension in its sharpest form: an honest remainder weakens your own headline, and silence lets a reader convert absence of findings into evidence of safety. Both costs are real. The job is to pay the first one, deliberately, and to know exactly which parts of the client's request you can accommodate. **Start by seeing what the sentence actually claims** “No critical issues found” is not false. It is *unbounded*, and it smuggles two things past the reader. The first is scope: it does not say two of eight categories were driven, so it reads as a statement about the system rather than about a quarter of a plan. The second is quieter and catches even experienced readers — “critical” is a severity band. A report can honestly say “no critical issues” while carrying eleven findings rated high, and once the sentence is quoted into a questionnaire, the band's definition does not travel with it. Two unstated denominators in six words. **Write the bound inside the sentence** Not in a footnote, not two pages later, not in an appendix: in the same sentence, because summaries are quoted one sentence at a time. “Within the two attack categories exercised (X and Y), no critical issues were identified; six of the eight planned categories were not exercised and are listed with reasons in section N.” Same finding, with the part that makes it interpretable restored. Then say plainly what the reader must not conclude: the six untested categories are unknown, not clear. **Give the remainder structure, not apology** For each unexercised category record four things: what it is, why it was not run (hours, environment, authorisation, a dependency that never landed), what follows if it does harbour a problem, and what closing it would take — in days and in spend. That last column is the one that changes the conversation: ```text Category Why not run To close Indirect injection (RAG) ingestion env unbuilt 2 d eng + ~$120 model calls Tool / function misuse tool creds not issued 1.5 d Multi-turn escalation hours exhausted 2 d, promptfoo numTests 50/plugin ``` Written that way the remainder stops being a confession and becomes the scoping document for the next engagement — which is usually what the person paying actually needs. It also makes visible that the hours bought a quarter of the plan, which is a procurement fact somebody should own. **Understand why the client is pushing** Usually the report is going to a board, an auditor or a customer security questionnaire, and the client needs *a* defensible sentence to paste. Give them one that is true and quotable. A bounded claim survives the follow-up question “what exactly did you test?”; an unbounded one collapses the first time anyone asks, and takes the tester's credibility with it — including on the findings that were real. **Where the line sits** Negotiable: ordering, section titles, tone, level of detail, whether the remainder is called “agreed limitations and planned follow-up”. Not negotiable: the remainder's presence, the scope clause inside the conclusion, and any wording that implies a denominator you did not test. If a client insists on removal, that is a sign-off conversation with whoever owns the engagement, not an editing decision — and refusing every change, including a retitle, is its own mistake, because it burns the relationship over a presentation question that has a solution. **Where the number misleads** Watch the amplification path. The summary is extracted into slides and questionnaire answers with qualifiers dropped; six months later a due-diligence reader sees “red-teamed, no critical issues” with no idea it described two categories. Two defences: keep the bound inside the quotable sentence so it travels with any excerpt, and add a short “what this report does not say” block near the front for the reader who reads only page one. And be alert to per-category arithmetic: any statistic computed across “all categories” when six were never driven has a denominator that never existed. **What I check before sending** Does every assurance sentence carry its own scope clause? Would each sentence still be honest quoted alone, out of the document? Does the summary name the untested categories, or only the appendix? Does the remainder carry cost-to-close, so it reads as a plan? And is any severity band used as if it were a verdict on the whole system?
- The client says the remainder list will scare their board. What do you offer?Re-title it as agreed limitations and planned follow-up, put each item's reason next to it, and add a costed scope for closing them. Same content, framed as a plan rather than a confession.
- Why keep the scope clause inside the conclusion sentence rather than in a nearby paragraph?Because summaries get quoted into slides and questionnaires one sentence at a time. A qualifier in the next paragraph does not travel; one inside the sentence does.
saying these in an interview costs you the question
- Agreeing to an unbounded 'no critical issues found' because it is technically true
- Burying the untested categories in an appendix the summary never references
- Claiming the untested categories are low risk with no basis
- Treating the client's discomfort as a reason to change the finding rather than its presentation