skip to content

Your scenario suite has drifted into a 47-minute interface-driven pack. How do you get feedback back?

level: seniorimportance: should knowfreq 54%

answer

  1. Wall clock is a symptom
  2. Group scenarios by the driver beneath them
  3. Sentences are independent of the driver
  4. Duplicated rules are the cheapest minutes back
  5. Independence before workers, budget after

basics

~20 s

Diagnose first: per-scenario durations, which layer each scenario drives, and how many scenarios restate the same rule. Then re-point most scenarios below the interface, delete duplicates, and keep only a thin set of interface-driven journeys.

solid answer

~50 s

The wall clock is the symptom; the cause is usually the level the scenarios run at and how many of them there are. I would group scenarios by the driver their steps use, look at the duration distribution rather than an average, and count scenarios per business rule. The strongest lever is that the readable sentences are independent of the driver beneath them: the same scenario text can be bound to the application boundary instead of the interface without changing a word a business reader sees, which typically removes most of the run time. Then delete restatements of rules already covered, express data variations at a lower level, and keep a deliberately small interface-driven set for the journeys where the interface itself carries the risk. Parallel running comes last, and only once each scenario owns its own data.

code

pseudocode · 5 lines
pseudocode
scenario_text = "points are credited once for a settled order"

binding(scenario_text, driver = interface_driver)
# same sentence, cheaper driver:
binding(scenario_text, driver = application_boundary_driver)

go deeper

for a junior

Know that scenarios can be bound to something other than the interface, and that a suite driving everything through the screen will be slow. You are not expected to plan the migration.

for a middle

Explain the mechanics of re-pointing: the sentences stay, the driver underneath changes, and data variations belong at a level where each case is cheap. Be able to describe what you would measure first.

for a senior

Show the diagnosis before the fix — duration distribution, driver grouping, scenarios per rule — and be honest about what you deleted and why parallel execution came last. Interviewers want the sequence, not a list of tactics.

for a principal

Own the policy that prevents recurrence: the time budget per trigger, who decides what earns a scenario, and how you separate the specification suite from the regression estate without leaving checks homeless.

## How a readable suite turns into a slow regression pack The drift is gradual and every step of it is locally reasonable. The first scenarios are written through the interface, because that is the most convincing end-to-end proof and the only driver the team has. Once that glue exists, the cheapest way to write the next scenario is to reuse it, so the next twenty also drive the interface. Then the suite becomes the only automation with a real owner, so regression checks that have nothing to do with business readability get added to it too, because that is where checks live. Nobody ever decided "our specification suite is now our slowest regression pack"; the decision was made twenty times, five minutes at a time. The eleven-person team on the loyalty-points ledger arrived there with 340 scenarios and a 47-minute run. Of those, 216 drove the interface; 74 restated a rule that another scenario already covered with different data; and a handful asserted internal detail no business reader would recognise. The symptom people complained about was the wall clock. The cause was the level the scenarios ran at and the number of them, not the runner. ## Diagnose before you optimise Collect four things, all from data the suite already produces: 1. **A per-scenario duration distribution**, not an average. A long tail of a few multi-minute scenarios is a different problem from a flat mass of thirty-second ones, and the remedies differ. 2. **Which layer each scenario drives.** Group them by the driver their steps use — interface, application boundary, domain directly. This is usually a short grouping over the glue. 3. **Scenarios per business rule.** More than one or two scenarios naming the same rule is duplicated coverage wearing different data. 4. **Which scenarios have a business reader.** A scenario asserting a technical artefact is a regression check that happens to be written in sentences; it is paying the translation cost and collecting none of the benefit. ## The lever this kind of suite uniquely has The readable half of the suite is independent of the driver underneath it. The same sentence — "points are credited once for a settled order" — can be bound to a driver that clicks through the interface, or to one that calls the application boundary directly, without changing a word a business reader sees. That is the strongest move available: re-point the bulk of the scenarios below the interface, keep a deliberately thin set of interface-driven journeys for the risks that only the interface carries, and leave the sentences alone. For the ledger team, moving the rule scenarios to the application boundary and keeping 12 interface-driven journeys brought the full pack from 47 minutes to about 11, before any parallelism was involved. ## The other three moves, in order of payoff **Delete duplication rather than optimise it.** The 74 restatements were the cheapest minutes to recover, and deleting a duplicate risks nothing that the surviving scenario does not still assert. Where genuinely many data variations matter, express the rule once at the readable level and drive the variations at a lower level, where each case costs milliseconds. **Make scenarios independent, then run them in parallel.** Independence means each scenario creates the data it needs and leaves nothing behind — two scenarios mutating the same ledger account will interfere, and interference looks exactly like a product defect, so investigation cost eats the time that parallelism saved. Independence first, workers second. **Split by trigger.** With a labelled subset the commit stage runs a few minutes' worth and the full pack runs on a slower cadence, so the wall-clock number stops being one number serving every audience. ## Keeping it from coming back The drift returns unless something holds the line. Three guards work: a declared time budget per trigger, enforced by the build rather than by good intentions; a review question on every new scenario — "which layer must this run at, and what does running it higher up buy?"; and a standing rule that a new scenario is added for a new business rule, while a new regression check goes wherever regression checks belong. A specification suite that grows only with the rule set stays a specification. One that grows with every check the team can think of becomes a slow pack again, and the next person will have this same conversation in eighteen months.

  • Which scenarios do you deliberately keep running through the interface?
    The ones where the interface itself carries the risk: the primary journeys a customer completes end to end, anything whose wiring across boundaries has broken before, and anything a stakeholder will not believe unless they see it exercised through the real surface. That is normally a small, named set with an owner, not a residue of whatever nobody got round to moving.
  • How do you stop the suite drifting back once you have cut it?
    Put a declared time budget on each trigger and enforce it in the build, so growth fails loudly rather than accumulating. Add a review question on every new scenario about which layer it must run at. And keep the distinction that a new business rule earns a scenario while a new regression check goes wherever checks belong; a specification suite that absorbs every check becomes a slow pack again within a year.
  • How do you justify deleting scenarios to a stakeholder who reads the suite as the specification?
    Show that you are deleting restatements, not rules: for each deletion, name the surviving scenario that still asserts the same rule. Deleting duplicated coverage does not change what the suite claims, and it makes the remaining document readable. If a deletion would remove the only statement of a rule, it is not a duplicate and it stays, whatever it costs to run.

saying these in an interview costs you the question

  • Adds parallel workers before scenarios own their data
  • Optimises waits instead of changing the level scenarios run at
  • Assumes only interface-driven scenarios prove anything
  • Keeps duplicated rule coverage as extra safety
  • Reports an average duration instead of the distribution
  • Treats the suite as the home for every regression check

context

resolving links…