A stored manual case definition has ten steps, and the automated test claiming it exercises only the first four. How do you represent that partial automation honestly?
answer
- the definition is the unit counted
- a binary mark must lie somewhere
- split along a real seam
- over-claiming removes work from the schedule
basics
~20 sEither split the definition so each part is wholly automated or wholly manual, or keep one definition in an explicit partial state that records what the automation asserts and what a human still checks. Marking the whole definition automated is the one wrong answer.
solid answer
~50 sPartial automation is a modelling problem, not a reporting one. A definition is the unit the repository counts, so a definition that is half covered forces a binary mark to lie in one direction or the other. The cleanest fix is to **split the definition** along the seam the automation actually reaches, so each resulting definition is honestly automated or honestly manual, and each gets its own outcome. Where the steps genuinely form one indivisible flow, keep one definition but put it in an explicit partial state, with a written note naming which assertions the automated test makes and which checks a human still performs. Whichever you pick, the automated test must not claim the definition as fully covered, and the manual half must remain visible in someone's work queue — the real damage of a false automated mark is that the manual steps stop being scheduled at all.
go deeper
Know that a stored definition is marked automated as a whole, so a test covering only some of its steps cannot honestly claim it without something else recording what is left.
Explain the two mechanisms available — splitting the definition along the automation seam, or an explicit partial state with a note — and what each does to how the definition is counted.
Show the asymmetry: under-claiming wastes visible effort, over-claiming silently removes work from the schedule. Then say how you decide by asking what actually stops the automation.
Own the decay problem. Set the rhythm for reviewing partial records, decide who may declare one, and make sure every consumer of the automation figure knows how the partial category is counted.
## Why partial automation is awkward A case repository counts **definitions**, not steps. Automated or not is a property the product attaches to the whole record, because the whole record is what gets a result. So the moment an automated test covers four of ten steps, a binary mark has to lie: mark it automated and six steps quietly stop being anyone's job; leave it manual and the four automated steps are re-run by hand forever and count for nothing. The damage is asymmetric, and it is worth naming plainly. An under-claimed definition wastes effort, which is visible — someone notices they are re-doing work a machine did. An over-claimed definition **removes work from the schedule**, which is invisible until the untested behaviour ships broken. Whatever you choose, choose it in the direction that keeps the remaining manual work in a queue somebody reads. ## Three honest representations 1. **Split the definition along the automation seam.** Turn one ten-step definition into a four-step definition, wholly automated, and a six-step definition, wholly manual. Each gets its own identifier, its own outcome, and its own honest contribution to the automation figure. This is usually right, because if automation stops cleanly at step four, that boundary is real — it is generally where the flow leaves what the harness can drive. 2. **Keep one definition and mark it partial.** Most repositories can carry the state with a field of their own, and where they cannot, a convention plus a note does the job. What matters is that the record says, in words a person can read, which assertions the automation makes and which checks remain human. The cost is that the automation figure now has a third category and every consumer of the figure has to be told how it is counted. 3. **Keep it manual, with the automation as supporting evidence.** The automated test still runs and still reports, but the definition stays in the manual population and the automated result is context for whoever executes it. This is the right answer when the automated part is a smoke check on setup rather than a real verification of the definition's expected result. | Representation | Automation figure | Manual work stays visible | Main cost | |---|---|---|---| | Split into two definitions | Exactly right | Yes, as its own record | Renumbering, and links to the old definition to redirect | | One definition, partial state | Needs a third category | Yes, if the note is read | Every consumer must understand the category | | Manual, automation as evidence | Understates coverage | Yes | Automated work earns no credit | | Mark the whole thing automated | Overstates coverage | **No** | Six steps silently leave the schedule | ## Choosing between them The deciding question is whether the seam is real. Ask what stops the automation at step four: - **It stops because the flow leaves the system** — a physical device, a third-party sandbox nobody can drive, a visual judgement, an approval by a real person. The seam is structural and durable, so **split**. - **It stops because nobody has written the rest yet.** The seam is temporary and will move next sprint. Splitting now creates two records you will re-merge, so **keep one definition in a partial state** and let the note record the intent. - **It stops because the four steps are only setup.** The automation is not verifying the definition's expected result at all, so it earns no coverage credit; keep the definition manual. ## What the pairing must not do - **Do not let the test declare the definition as fully covered.** Whatever state the record carries, the declaration in code is what the push acts on, and a full claim on a partial test is how the figure goes wrong at source. - **Do not encode the partiality only in the test's own name or a comment.** Nothing reads it. The repository is where the manual half has to remain visible, because that is where work gets scheduled. - **Do not let a partial automated pass become the definition's last recorded outcome without qualification.** A green result against four of ten steps reads exactly like a green result against ten, and the difference lives only in the record's state. - **Do not leave the split half-finished.** If you split, redirect what pointed at the original definition and retire the original, so a single behaviour is not represented twice with contradictory outcomes. ## The reconciliation habit Partial automation is not a one-off decision; it decays. The automation grows, the definition's steps get rewritten, and a partial record drifts toward being either fully automated or wrongly claimed. Review the partial population on a fixed rhythm and ask one question of each: *is this still partial, and is the manual remainder still being executed by somebody?* A partial record that nobody has executed manually for months is not partial — it is an unmarked coverage gap wearing a careful label.
- You split the definition in two. What must happen to the original record?Retire it and redirect everything that pointed at it, so one behaviour is not represented twice with conflicting outcomes. Anything declaring the old identifier has to be updated to declare the correct half, and the retired record should resolve to something a reader can follow rather than simply vanishing from the tree.
- How do you stop a partial record from quietly becoming a permanent one?Review the partial population on a fixed rhythm and ask of each whether the manual remainder is still actually being executed. A partial record nobody has run by hand for months is not partial — it is an unmarked gap with a reassuring label, and it should either be finished or reclassified.
saying these in an interview costs you the question
- Marks the whole definition automated because a test claims it
- Records the partiality only in a code comment
- Assumes a green partial run means the definition passed
- Splits every definition without checking the seam is real
- Leaves the original record live after splitting it