What belongs in a smoke suite that gates a build, and what disqualifies a case from it?
answer
- Cheap question asked before expensive work
- Breadth over depth, one per path
- Fix the runtime budget first
- Failure must invalidate later results
- Blocking, or it becomes decoration
basics
~20 sA build-gating smoke suite is a short, broad set of checks that decides whether a build is worth testing further. Include one fast, deterministic case per critical path. Exclude slow, data-hungry, flaky or deep edge-case checks.
solid answer
~50 sA smoke suite is breadth-first and shallow: it touches each path whose failure would invalidate every later result — the service starts, configuration and reference data load, the main transaction completes, the store is reachable — and asserts only enough to see that the path is alive. Fix the runtime budget first, in minutes, and admit cases only if they fit it. A case is disqualified if it is slow, needs scarce or hand-made data, depends on shared mutable state, asserts values that change often, or is nondeterministic on unchanged code, because a gate that fails at random teaches people to re-run it until it is green. The gate must block: if smoke fails, the build is not testable and nothing downstream runs. Green smoke means the build is worth testing, never that it is tested.
code
pseudocode · 13 linesSMOKE_BUDGET = seconds(280)
def admit_to_smoke(case, subset):
if case.median_runtime > seconds(45): return False
if case.flake_rate_on_unchanged_code > 0: return False
if case.needs_prepared_data: return False
if case.critical_path in subset.paths: return False # no duplicate signal
if subset.runtime + case.median_runtime > SMOKE_BUDGET: return False
return case.failure_invalidates_downstream_results
on_smoke_failure:
stop_pipeline() # nothing downstream runs
do_not_retry() # diagnose the build, not the gatego deeper
Be ready to say what a smoke run is for in one sentence — deciding whether a build is worth testing further — and to give two or three examples of paths it would touch, such as startup, reference-data load and one main transaction.
Explain the admission rules and the runtime budget: one case per critical path, seconds not minutes, deterministic, self-sufficient data, stable assertions. Expect to name what disqualifies a case and why the gate must block rather than report.
Show how you keep the subset from decaying: defending a budget against the after-incident urge to add cases, removing a nondeterministic case from the gate instead of bypassing the gate, and stating plainly what a green run does and does not license.
Own the policy question of what the organisation is allowed to spend on every single build, who may add to the gate, and what a failure obliges people to do. Be ready to argue that an advisory gate provides no confidence at all.
## What the subset is for A build-gating smoke suite exists to answer one cheap question before anyone spends expensive time: **is this build worth testing at all?** It is deliberately broad and deliberately shallow. It walks each path whose failure would make every later result meaningless, and it asserts only enough to show that the path is alive. A related term, **sanity check**, is often used for the opposite shape — a narrow, deeper look at one area after a small change. Usage genuinely varies between organisations, and plenty of teams use the two words interchangeably; if an interviewer leans on the distinction, say how your team used them rather than asserting a universal definition. ## Admission criteria Decide the budget first — a number, in minutes, that the team will actually tolerate on every build — and then fit cases into it. Say the budget is 4 minutes 40 seconds. A case earns a place if it satisfies all of: - **It covers a distinct critical path.** Startup, configuration and reference-data load, the main transaction, the persistence path, an outbound integration reachable. One case per path; a second case on the same path adds runtime and no new signal. - **Its failure invalidates later results.** If the reference data does not load, every downstream result is noise. That is the real admission test. - **It is fast.** Seconds, not minutes. - **It is deterministic on unchanged code.** A gate that fails once in forty runs on unchanged code is worse than no gate, because the response becomes "re-run it". - **It is self-sufficient.** It builds and tears down its own data, depends on no hand-prepared record, and does not collide with a parallel run over shared mutable state. - **Its assertions are stable.** Assert that a fare was returned and the ledger recorded the journey, not that the fare equals a figure that changes whenever the tariff table is updated. ## Disqualifiers Slow checks; anything needing a hand-built dataset or an operator step; deep edge cases such as a cap crossing midnight, a concession matrix, or proration of a 31-day pass — those are pack material; anything nondeterministic; anything whose assertions churn with ordinary content changes; and any case that duplicates the signal of a case already in the subset. ## A worked example For a fare calculator on a public-transport network, a defensible gating subset is roughly: the service starts and reports ready; the current tariff table loads and reports a version; one single-journey fare returns a value; one card tap is recorded as a journey; the ledger store accepts a write and reads it back. Five cases, around three and a half minutes. Everything about capping rules, concessions, zone boundaries and multi-day passes belongs in the regression pack, not here. Notice what the subset would **not** have caught in a real incident on that system: a fix that made each capped day write nine refund-ledger entries instead of one. The gating subset checks that the ledger accepts a write, not that a capped day writes exactly one entry. That is correct behaviour for a smoke subset — it is a filter, not a verdict — and pretending otherwise is how the subset grows. ## How the subset decays Two decay patterns dominate, and interviewers like hearing you name them. **Growth.** Every incident produces the suggestion "add it to smoke". Six minutes becomes twenty-six, the gate stops being fast feedback, and every build pays. The discipline is the budget: a new case enters only if it covers a path nothing else in the subset covers, and it must fit. Anything else goes into the pack. **Demotion to advisory.** A gate that fails intermittently gets made non-blocking to unblock people. From that moment it reports rather than gates, and within weeks nobody reads it. Either the case is stable enough to block, or it does not belong in the gating subset. ## What a failure means A smoke failure stops the line: nothing downstream runs, because results from a build that cannot start or load its reference data are noise. The response is to diagnose the build, not to re-run the gate hoping for green. If a team habitually re-runs it, that is direct evidence the subset contains a case that does not meet the determinism bar. ## What green means Green smoke licenses further testing. It is not a statement about quality, correctness or release readiness, and describing it that way in an interview is the single most common way candidates lose the question.
- Your build-gating smoke suite has grown from five minutes to twenty-six. What do you do?Re-derive it from the budget rather than trimming ad hoc. List the critical paths whose failure invalidates downstream results, keep one case per path, and hold the total under the number the team will tolerate on every build. Everything that arrived by way of "add it to smoke after that incident" moves into the regression pack, where it still runs, just not on the gate. Then defend the budget as a standing constraint, because without one the same growth restarts immediately.
- A gating smoke case fails about once in forty runs on unchanged code. What is the right response?Take it out of the gate. A nondeterministic case on a blocking gate trains everyone to re-run until green, which destroys the gate's meaning for every other case in it. Move it out, keep it running where a failure is investigated rather than bypassed, and fix the source of the nondeterminism before it comes back. Making the whole gate advisory to accommodate one case is the worse trade: it costs you the signal from the four cases that were reliable.
- Can a smoke subset be run against production rather than a build?Yes, and the shape carries over: a short, broad set of shallow checks confirming the deployed system's critical paths are alive after a release. The admission rules tighten, because the checks now run against real data and real side effects — read-only or self-cleaning cases only, no test records left behind, and no action a real user would be billed for. The purpose is the same: decide quickly whether the deployment is healthy enough to trust.
A smoke suite is the pre-flight walkaround: doors, tyres, control surfaces, fuel. It is fast and shallow on purpose, and passing it means the aircraft is worth boarding, not that the flight will be smooth.
saying these in an interview costs you the question
- Describes a smoke suite as a deep check of every business rule
- Lets the build-gating subset grow to tens of minutes
- Keeps a case that fails at random and re-runs the gate
- Makes the gate advisory so nobody is blocked
- Says a green smoke run means the build is tested