Your SOC closes 400 alerts a day but its case notes are unusable a year later - how do you fix that without halving throughput?
answer
- not every case deserves equal depth
- make the tool write the boilerplate
- a skeleton, not a twenty-field template
- sample and re-read closed cases
- test: can a peer re-derive the verdict
basics
~20 sTier note depth by disposition, capture from the tooling what analysts should not retype, mandate a short skeleton rather than a long template, and sample closed cases to test whether a second analyst reaches the same verdict from the note alone.
solid answer
~50 sI would not ask 400 cases a day to carry the same note. Most bulk dispositions need structured fields the console can populate itself - entities, rule identity, query text, UTC timestamps - and the analyst adds one line of reasoning. The cases that turn on human judgment, anything escalated and anything closed as benign on an analyst's say-so, get the full record: scope searched, what was ruled out, why. Second, make the tooling do the retyping; note quality collapses when the standard is a form rather than a by-product of the work. Third, replace the long template with a four-item skeleton - hypothesis, what was checked and over what window, what was ruled out, verdict and reason. Finally, measure it honestly: sample closed cases weekly and have a peer try to reach the same verdict from the note alone. Report that at team level, not per analyst, or the metric gets gamed within a month.
go deeper
You will not set this standard, but know what it asks of you: the depth of a note follows the decision behind it, and a judgment close needs more than a bulk close.
Be able to argue why a long mandatory template fails under queue pressure and what a short skeleton captures instead - hypothesis, scope checked, ruled out, verdict and reason.
Show how you would raise quality in practice: tooling that populates the mechanical fields, sampled peer re-derivation as the acceptance test, and depth tiered by disposition.
Own the tradeoff explicitly - analyst-minutes against retrospective defensibility - along with who owns the standard, what retention obligations dictate, and how you refuse the accretion of mandatory fields.
## Diagnose before prescribing `The notes are unusable` is a symptom with three quite different causes, and the fix depends on which one you have: - **No standard.** Nobody ever said what a closed case must carry, so quality tracks individual conscientiousness. - **A standard nobody can afford.** A long template exists; analysts have four minutes per alert; the template is filled with `n/a` and the standard is fiction. - **A tooling problem.** Analysts are retyping what the console already knows, so the note is a transcription chore rather than a record of thinking. Most SOCs with 400 closes a day have the second and third. Writing a stricter policy is the response that fails, because the constraint is analyst-minutes and policy does not create any. ## Tier the depth to the decision Not every case deserves the same record, and pretending otherwise is what makes the standard unaffordable. A workable split: - **Bulk and automated dispositions** - alerts closed by an enrichment rule or an obvious, repeatedly-seen benign pattern. These need structured fields, populated by the tooling, plus a single reason code. Prose adds nothing a query cannot recover. - **Analyst-judgment closes** - anything where a human decided that activity which fired a rule is acceptable, especially a benign true positive where the rule was right and the activity was authorised. These need the full record, because this is the category that gets re-read after something goes wrong, and the question will be whether the decision was reasonable on what was known. - **Escalated cases** - the full record plus the handover discipline, since they will be read by people who were not there. The honest framing for leadership: you are buying auditability for the cases where it can be needed, and paying for it with minutes taken from the cases where it cannot. ## Make the tool write the boilerplate Everything mechanical should arrive in the case without a human keystroke: the entities, the rule identifier and version, the query text an analyst ran in the console, the artefacts pulled, timestamps normalised to UTC on capture rather than by hand. Every field a human must retype is a field that will be wrong, blank, or in local time. This is usually the highest-leverage change available and it is an engineering investment, not a policy one - which means it needs a budget line and an owner, and that is the actual decision being asked of a lead. ## A skeleton, not a template Four prompts, in the case form, that mirror how the work is actually done: what did you think was happening; what did you check and over what window and sources; what did you rule out and how; what is the verdict and why. That is short enough to be completed under time pressure and long enough to pass the only test that matters. A twenty-field template collects compliance; a four-field skeleton collects reasoning. ## Measure the thing, not a proxy The acceptance test for a case note is re-derivation: give a peer only the note and ask them to reach a verdict. Sample a small number of closed cases per week across dispositions and analysts, run that test, and record agreement plus the reason for each disagreement. It is slow and it is the only measure that resists gaming. The proxies all fail predictably. Mandatory-field completion rates measure form-filling; note length is gamed the day it is published; time-spent-per-note punishes the efficient. And report results at team level, framed as a signal about the standard and the tooling rather than a scorecard on individuals - the moment note quality is a personal performance metric, notes get longer and no more useful. ## The organisational parts **Ownership.** Someone owns the standard, is allowed to change it, and is allowed to say no when a new mandatory field is proposed. Standards drift upward by accretion because every addition is individually reasonable. **Retention and the audit obligation.** How long closed cases must be readable, in what form, and by whom, is usually an external requirement rather than a SOC preference. Find out what it actually is before designing the record - it determines what has to be inside the case rather than linked from it, because linked consoles and chat systems have their own, shorter, lifetimes. **The existing backlog.** Do not retro-fix a year of thin notes. Declare a cutover, keep what exists, and state the limitation plainly. Reconstructing case notes from memory produces a record that is confidently wrong, which in an audit or a legal context is worse than an admitted gap. **Expectation setting.** Say out loud what the change costs - a small number of analyst-minutes per judgment close - and what it buys: the ability to answer, eleven months later, why an alert was closed, with the person who closed it long gone.
- How do you measure note quality in a way analysts cannot game?Blind re-derivation on a sample: a peer reads only the note and states the verdict they would reach, and disagreements are reviewed for cause. Report it at team level as a signal about the standard and the tooling. Field-completion rates, note length and time-per-note are all gamed within weeks because each rewards volume rather than reasoning.
- An auditor asks for evidence that closed alerts were properly reviewed. What do you show?The written standard, the sampling review and its results over time including the disagreements and what changed as a result, and a set of real closed cases whose notes carry scope, evidence and reasoning. A process with a measurement and a feedback loop is far stronger evidence than an assertion that everyone writes good notes.
- What do you do about the year of thin notes already written?Leave them. Declare a cutover date, preserve what exists under the retention rules, and state the limitation explicitly when anyone relies on that period. Reconstructing notes from memory a year later produces confident fiction, which is worse than an acknowledged gap in front of an auditor or in any later dispute.
saying these in an interview costs you the question
- Mandates a long template for every closed alert
- Measures note quality by length or word count
- Blames individual analysts, not the standard or tooling
- Assumes every closed alert deserves the same depth
- Retro-writes missing notes from memory
- Adds mandatory fields with no owner able to refuse