skip to content

In a test that drives the application's screens, what should the final assertion check so the case is a real oracle?

level: middleimportance: should knowfreq 57%

answer

  1. Ask what a person would notice
  2. Would this still pass if broken?
  3. Existence proves nothing; content does
  4. Too strict red for harmless changes
  5. Break it on purpose and watch

basics

~20 s

Assert the outcome a person would notice - the value, message or state the screen now shows - and assert its content, not just that an element exists. A case that cannot fail when the feature is broken is not an oracle.

solid answer

~50 s

The oracle is whatever decides the behaviour was wrong, and at screen level that should be the thing a user would notice: the fare displayed, the confirmation shown, the row that appears with the right values. Two failure modes bracket the good answer. Assert too little - an element exists, a page loaded, no error was thrown - and the case passes while the feature is broken. Assert too much - exact markup, layout, every field on the screen - and it fails on harmless changes and stops being trusted. So assert the specific value the behaviour produces, keep unrelated checks out of the case, and if the claim includes persistence, re-read the screen after a reload rather than trusting what is on it. The quickest sanity check is to break the behaviour deliberately and confirm the case goes red.

code

pseudocode · 10 lines
pseudocode
// weak oracle: green even when the fare is wrong
assert isVisible(farePanel)

// sharp oracle: named value a rider would read
assert displayedFare == "3.40"
assert displayedFareBasis contains "peak"

// persistence claimed, so re-read after a reload
reload()
assert displayedFare == "3.40"

go deeper

for a junior

Be ready to say that checking an element exists is not checking behaviour, and to name a better assertion: the actual value or message a person would read after the action.

for a middle

An interviewer expects both failure modes and the line between them - too weak to catch a real break, too strict to survive a harmless change - plus the habit of computing an expected value from the rule instead of from a passing run.

for a senior

Show how you prove an oracle works: break the behaviour deliberately, check the case goes red, and check the message names the difference. Talk about how vague assertions and noisy over-specification both erode trust in a pack.

for a principal

Own the standard for what a screen-level case is allowed to assert, and the trade-off it encodes: precision buys trust and diagnosis, breadth buys accidental catches at the price of churn. Be able to defend where you drew that line.

### What an oracle is An **oracle** is the thing that decides a behaviour is wrong. Every case has one, explicitly or by accident, and the quality of a case is mostly the quality of its oracle. At screen level the natural oracle is the user's own: after this journey, what would a person see that tells them it worked? The fare shown for the zones they chose. The confirmation naming what they bought. The list containing the entry they just added, with the right values in it. ### The two ways an oracle goes wrong **Too weak.** The case asserts something that would still be true if the feature were completely broken. Classic weak oracles at screen level: - *An element is present.* A fare panel that renders with an empty or stale value still satisfies "the fare element exists". - *The page loaded / no error appeared.* This is a crash detector, not a behaviour check. - *A request was sent.* The behaviour is what the system did with the answer, not that it asked. - *A value changed.* Changing to the wrong value passes. - *A count.* Three rows appeared, but nothing says they are the right three. On a public-transport fare calculator, a case that selects zone 2 to zone 5 at 08:12 with a senior concession and then asserts only that a fare is displayed will keep passing after a rounding change turns 3.40 into 3.4083, after the concession stops applying, and after peak pricing is skipped entirely. It is green, it costs run time, and it protects nothing. **Too strong.** The opposite failure is over-specification: asserting the whole rendered region, the exact wording of every label, positions, or a full snapshot of the screen. Now every copy edit or layout change turns the pack red for reasons nobody cares about, and the team starts updating expectations without reading them. That is how a screen suite loses trust - not by missing bugs, but by crying wolf until red stops meaning anything. The target between them: **assert the specific, user-meaningful value the behaviour under test produces**, and nothing else. ### Making the oracle sharp A few habits do most of the work: 1. **Assert content, not existence.** Not "a fare is shown" but "the fare shown is 3.40". A value assertion fails when the calculation is wrong; an existence assertion cannot. 2. **Assert what the requirement actually claims.** If the rule is that a senior concession reduces a peak fare, the case should show a fare that differs from the unconcessioned one - ideally by checking the specific expected figure, computed by hand from the rule rather than copied from a passing run. 3. **Do not copy the current output into the expectation.** Recording what the system printed and asserting it back makes the case agree with whatever the system does, including whatever it does wrong. 4. **Include the persistence claim when there is one.** If the behaviour is meant to save something, reload and re-read. What is on screen straight after a click can be optimistic and not yet stored. 5. **Keep unrelated assertions out.** A case that also checks the footer year, a banner and a navigation highlight now fails for three reasons that have nothing to do with its name. Those checks belong to their own cases, or to a lower level. 6. **Prefer the wording a person would use.** Assert on what is presented - the visible message, the labelled value, the state a screen reader would announce - so the assertion survives internal restructuring that changes nothing a user perceives. ### Testing the test The most convincing thing a candidate can say is that they verified the oracle. Break the behaviour on purpose - hard-code the wrong fare, disable the concession rule - and confirm the case fails, and fails with a message that names the difference. A case that stays green under a deliberate break is decoration. This is the cheap, manual version of the same idea a mutation-analysis tool automates. The failure message matters as much as the verdict. "Expected fare 3.40 but the screen showed 4.85" ends the investigation. "Assertion failed" starts one. ### A worked contrast ```pseudocode // weak: passes with an empty, stale or wrong fare assert isVisible(farePanel) // over-specified: fails on a copy edit or a layout change assert wholeRenderedRegion(farePanel) == "Fare 3.40 peak (senior concession)" // sharp: fails exactly when the fare is wrong assert displayedFare == "3.40" assert displayedFareBasis contains "peak" ``` ### How many assertions One behaviour, one claim - but a claim can legitimately need two or three related checks, as above, where the fare and the basis for it together express the rule. The test to apply is whether every assertion in the case would be *interesting* if it failed. If one of them failing would make you shrug and delete it, it should not be there. Where several related checks belong together and you want all their results in one run, an accumulating (soft) assertion style reports every mismatch instead of stopping at the first - useful for a form with many fields, and a poor excuse for stuffing unrelated claims into one case.

  • How would you convince yourself an assertion is actually an oracle rather than decoration?
    Break the behaviour on purpose and re-run. Hard-code a wrong value, or switch off the rule the case is about, and confirm the case goes red and the message names the difference. If it stays green, the assertion is not checking the behaviour. That deliberate-break habit is the manual version of what mutation analysis automates, and it costs a minute per case.
  • Where does a full snapshot of a screen region belong, if anywhere?
    It is a blunt oracle: it catches everything and explains nothing, and it goes red on changes nobody cares about. It earns a place where the whole rendering genuinely is the contract and churn is low. For a behaviour with a specific expected outcome, a named value assertion fails more precisely and survives unrelated edits.
  • A case asserts that a request was sent when the user submits. What is missing?
    The outcome. Sending the request is the system asking a question; the behaviour is what it does with the answer. A case that stops at the request passes when the response is mishandled, rendered wrongly or not rendered at all. Assert what the screen shows afterwards, and keep the request check only if the request itself is the contract under test.

saying these in an interview costs you the question

  • Asserting an element exists and calling that a check
  • Treating no error thrown as a passing behaviour check
  • Copying the current output into the expected value
  • Asserting exact markup or full screen snapshots by default
  • Stuffing unrelated checks into one journey case
  • Never verifying the case fails when the feature breaks

context

resolving links…