Testing & Quality Disciplines

Arrange builds the fixture, inputs and collaborators the test needs. Act invokes the one behaviour under test and captures its result. Assert compares that outcome with the expected one. Given-When-Then names the same three phases.

Acceptance testing judges a change against the stated acceptance criteria of the requirement it implements, not against the shape of the code. It passes when every criterion is demonstrably met, and those criteria are agreed before the work starts.

An approval test runs the code, writes the output to a received artefact, and compares it against a previously approved artefact. Nobody types the expected value: a human reviews the output once and approves it, and the test then guards that exact output.

Test the edges, not the middle: 0 and 1 low, 60 and 61 high. Defects hide where a comparison flips from accept to reject, so each limit gets a value on it and one just past.

A consumer-driven contract records the requests one consumer sends and the response parts it actually reads, then asserts the provider can still produce them. It checks the shape of the boundary, not whether the provider's answers are correct.

Emergent design means the structure of the code is an output of the cycle, not a plan made before it. Each restructuring step reshapes working code, so the design arrives gradually instead of being guessed up front.

Outside-in TDD starts at an outer boundary and invents collaborator roles as they are needed, standing them in until they are built. Inside-out TDD starts with domain behaviour built from real objects and wires the outer layers in last.

Watching the test fail proves the assertion can actually detect the behavior's absence. A test that has never been red may be mis-wired, never executed, or trivially true, and would report green forever while guarding nothing.

Because a green run after every move tells you exactly which move changed behaviour. Run once at the end and a red suite only says something in the last hour was wrong, so you debug instead of undoing.

Test-first means writing an executable test for behaviour that does not exist yet, then writing code to satisfy it. Test-after writes tests against finished code, so the cases are shaped by what the code already does.

The outer loop is a failing acceptance scenario written in the user's language; the inner loop is the fast unit cycle beneath it. The inner loop goes green many times over; the outer scenario goes green last, when the behaviour is complete.

A step definition is a function holding the automation code for one scenario step. The runner matches the step's plain-language text against a pattern registered with that function, converts any captured values into arguments, and calls it.

A Background holds context steps that run again before every scenario in that file, so shared setup is written once. Every scenario inherits those steps, whether or not it needs them.

Given states the context that already holds before the behaviour under test. When names the single event that triggers it. Then states the observable outcome that must follow. All three describe behaviour, not interface mechanics.

The document is generated from scenarios that actually executed, so it describes only behaviour the suite exercised. When the system changes and the scenario does not, the scenario fails and the page visibly breaks instead of quietly going stale.

A static-analysis baseline is a recorded snapshot of the findings an analyser already reports on existing code. The gate ignores those and fails only on findings outside the snapshot, so an old codebase can adopt analysis without a mass cleanup first.

Each placement buys something different: the editor gives feedback in seconds while you type, the pre-commit check keeps a whole commit clean, and the build is the only placement that runs for everyone and can actually block a merge.

A formatting rule governs how code looks - spacing, indentation, ordering - and never changes behaviour. A correctness rule flags a shape that is likely a bug: a discarded return value, an unreachable branch, a resource never released.

A source is where untrusted input enters the program, a sink is a sensitive operation that must not receive it, and a sanitizer neutralises the data. A finding is a source-to-sink path with no sanitizer on it.

A static type checker proves, without running the program, that every operation receives a value whose declared or inferred type it accepts. It rules out type-mismatch errors on all code paths, but says nothing about whether the logic is right.

Entry criteria are the conditions that must hold before a test cycle starts: a deployed build with a known change list, a healthy environment, seeded data, and agreed acceptance criteria. They prevent a cycle from burning its budget on setup failures.

Verification asks whether the product was built according to its specification. Validation asks whether that specification described the right thing to build. One checks conformance to a written statement, the other checks fitness for a real need.

A failed case ran and gave the wrong result, so it is evidence about the product. A blocked case never ran, because something stopped it. Merging the two hides untested scope and blames the product for an environment problem.

Confirmation testing re-runs the case that exposed the defect, to prove the fix works. Regression testing re-runs other cases around the change, to prove the fix broke nothing that previously passed. Both are needed after a fix.

Risk-based test selection ranks each area by how likely it is to fail and how badly a failure would hurt, then spends the deepest testing on the highest-ranked areas and the least on the lowest.

An unambiguous one-line title, the exact build identifier and environment, numbered preconditions and steps, the expected result and the actual result written separately, and evidence such as logs, screenshots or traces.

Defect density is a count of confirmed defects divided by a size unit — a thousand lines of code, a functional size unit, a module, a feature. Change the unit and the same product scores differently.

Origin phase is where the defect was introduced - the requirement, design, code, data or configuration work that created it. Escape phase is the last check that should have caught it and did not. Every defect has both.

The symptom is what was observed. The condition that produced it is the specific state or input that made the code fail that time. The weakness is the missing guard or assumption that allowed the whole family of failures.

Severity measures a defect's technical impact — how badly the product's behaviour breaks. Priority measures business urgency — how soon it should be fixed relative to other work. Different people set them, and the two values move independently.

Walk each step of the happy path and ask what an actor could do instead: skip it, repeat it, reorder it, supply another actor's identifier, or exceed a limit. Each answer becomes a case the system must refuse.

Volume testing keeps the request rate fixed and grows the stored dataset instead: more rows, longer collections, larger files. It exposes defects that appear only at size, such as unbounded queries, memory that scales with result size, and jobs that outgrow their window.

An internationalization defect is in the product code: a string that was never extracted, a hardcoded date format, a container that cannot hold a longer word. A translation defect is in the text itself, wrong or missing wording.

Ramp-up is the stretch while offered load climbs to the target level; steady state is where that load is held constant; warm-up is the early portion where caches, pools and compiled code settle. Report the steady state only.

A support matrix is the published grid of platform combinations — operating system, version, engine, device class — where a product promises to work. Each listed cell is a commitment: a defect that reproduces there is one you owe a fix.

A short reproduction proves which conditions actually cause the failure. Every step you can delete without losing the failure is noise that slows diagnosis, invites a cannot-reproduce close, and hides the real trigger from whoever has to fix it.

A bug bash is a short, time-boxed event where many people — developers, support, product and testers — hunt defects in one shared build at once, each covering an assigned area and filing into one channel.

A test charter is a one- or two-sentence mission for a single exploratory session. The common shape names three things: what to explore, what to explore it with, and what information the session should discover.

A test oracle is whatever you use to decide that an observed behaviour is wrong: a written specification, a standard, a comparable product, an earlier version, or your own expectation. Each is an imperfect model, so it can be wrong too.

A session is one uninterrupted block of testing, commonly 60, 90 or 120 minutes, spent on a single charter and written up as one reviewable record. The session, not the test case, is the unit of work you plan and count.

Case count measures how much was written, not how much can be detected. Drafted cases usually re-walk journeys existing cases already walk and check outcomes already checked, so the set of failures the suite can catch may not grow.

A repair hides every change that broke the original anchor: a control that moved, was relabelled, lost its announced name, or was replaced by a different one. The suite reports the flow works while the interface changed.

Assert the status code, the few fields the case is actually about, and the response shape the contract promises — and on failure paths the error envelope's code. Whole-body equality breaks whenever an unrelated field changes.

Point the system's configured destination at a receiver the run stands up, then read the recorded request back from it. The receiver must be reachable from where the system runs, not only from the machine running the suite.

Assert on a settled invariant, a property that stays true once the flow has finished, such as a terminal status or a final total. Do not assert on a counter or an intermediate status that is true for only a moment.

An object mother exposes named ready-made instances, so a test asks for a case by name. A test-data builder starts from valid defaults and lets a test override only the fields that case depends on.

An in-memory database stand-in starts in milliseconds, needs no external service and resets cleanly between cases, so a suite runs fast anywhere. It hides every behaviour where the production engine differs: dialect, type coercion, collation, constraint enforcement and locking.

Hand-built fixtures state only the few records a case needs, so they read clearly and carry no privacy risk. Generated records buy volume and variety cheaply. Production-derived extracts show real shapes and skew, but must be cut down and masked first.

A failing assertion aborts the test body, so cleanup written after it never executes and the records it created leak into later tests. Register teardown as an after-each hook or a resource-scoped block so it runs on both exits.

Random data makes a failure depend on values nobody chose. Recording the seed and printing it in the failure output lets anyone re-run the identical values, so the failure can be reproduced and fixed instead of guessed at.

Fuzzing drives a program with large volumes of generated, malformed or mutated input. Because no per-input expected result exists, the oracle is the program's own bad behaviour: crashes, hangs, assertion failures, and reports from an instrumented build.

Model-based testing builds an explicit model of intended behaviour, usually a state machine, and derives cases from it automatically. A generator walks paths through the model; each walk becomes a test, and the model predicts the expected result.

A round-trip property asserts that transforming a value and then reversing the transformation returns the original value, for every generated input. It misses paired defects: when both directions are wrong in mirror-image ways, the trip still returns the input unchanged.

Dumb mutation flips bytes in existing inputs and knows nothing about the format. Grammar-aware generation builds well-formed inputs from a description of the format. Coverage-guided fuzzing uses an instrumented build to keep inputs that reach new code and mutate those further.

Assert a rule the output must satisfy rather than a specific value: an invariant or postcondition, idempotence, agreement with a simpler reference implementation, or a metamorphic relation between two runs on related inputs.

Read the intent first — what problem the change claims to solve — then the contract it exposes, then the edge cases and error paths, then the tests. That order settles whether the change is right before you argue about how it is written.

A production defect costs more because the code change is only a fraction of the work. Someone must notice it, reproduce it and re-learn code written weeks ago, and the team then pays for data repair, support and an unplanned release.

The driver types and handles the mechanics. The navigator stays a step ahead — unhandled cases, naming, the remaining plan — and keeps talking. Rotating the keyboard on a timer keeps both people active and able to explain the change.

Four shapes recur: an independent test group that verifies finished work, testers embedded in each delivery team, whole-team quality where every engineer tests and no dedicated tester exists, and a small coaching group that builds testing skill in others.

Flake rate is the share of runs or cases giving a different verdict on the same unchanged revision. Measure it by recording every attempt, not just the final one, and track the trend: an acceptable level depends on suite size.

Four delivery failures matter: nothing comes back inside the deadline, the call is refused for volume, the provider is unavailable, or the input exceeds what the step accepts. Each strands the user differently, so each earns its own test case.

An adjudicable acceptance criterion names one observable property of the response, says where to look for it, and fixes the decision rule in advance, so two testers reading the same response record the same pass or fail.

A criterion with no stated allowance is read as absolute, so the first response that misses it forces an unplanned argument. Agreeing the allowance up front makes a rare miss an expected, recorded outcome rather than a crisis.

Write the evidence at failure time, because a repeat may never reproduce it: the exact input, the material and instruction text given to the generative step, the model identifier and generation settings, the raw output, and a correlation id.

Five layers can own it: the ordinary product code around the generative step, the instruction text sent to it, the material supplied with it, the configuration, and the generative model's own capability ceiling. Attribute only after evidence separates them.