skip to content

In Selenium, why does a test asserting that a violation banner has disappeared pass even when it is still displayed?

level: seniorimportance: must knowfreq 68%

answer

  1. A green absence check proves little
  2. Nothing found has several possible causes
  3. Prove the locator while it is present
  4. Catching Exception hides an invalid selector
  5. isEmpty on the plural call, not try/catch

basics

~20 s

Absence and a broken locator look identical to Selenium: both produce no match. Check absence with findElements and isEmpty, and prove the same locator matches while the banner is present, or the assertion passes for the wrong reason.

solid answer

~40 s

A lookup only answers whether anything matches right now, so a removed banner, a renamed class and a typo in the selector are indistinguishable — each returns an empty list, or throws from the singular call. For a positive assertion that is harmless because a broken locator fails the test; for a negative one it is dangerous, because a broken locator makes it pass. The fix is to make the locator prove itself: assert that the same locator matches while the banner is still on the food-safety inspection form, then assert `driver.findElements(banner).isEmpty()` after the correction. Use the plural call rather than a caught exception, and if you do catch, catch exactly `NoSuchElementException` — `catch (Exception e)` turns an invalid selector into a passing absence check.

code

java · 13 lines
java
By banner = By.cssSelector("#inspection-form .critical-violation-banner");

// Prove the locator can see the banner while it is still supposed to be there.
if (driver.findElements(banner).isEmpty()) {
  throw new IllegalStateException("setup failed: " + banner + " matched nothing");
}

driver.findElement(By.id("mark-violation-corrected")).click();

// Now an empty list means removal rather than a stale selector.
if (!driver.findElements(banner).isEmpty()) {
  throw new AssertionError(banner + " is still on the inspection form");
}

go deeper

for a junior

Know that checking something is gone is done with the plural lookup and an empty-list check, not by catching an exception from the singular one.

for a middle

Explain why an empty match list has several possible causes, and why a too-wide catch around a lookup turns unrelated errors into a passing absence assertion.

for a senior

Show the production judgment: pair every negative assertion with a proof that the locator matches while the element exists, and be able to describe an absence check you found had been green for months for the wrong reason.

for a principal

Own the convention across a suite: negative assertions are the ones that rot silently, so decide how they must be written and what evidence a review should demand before one is merged.

## Absence and a broken locator are the same observation An element lookup answers one question: **does anything in this search context match this locator right now?** It cannot tell you *why* the answer is no. A banner that was correctly removed, a class that the front end renamed last week, a typo in your selector, a search issued against the wrong document — all of them produce the identical result: an empty match list, or `NoSuchElementException` from the singular call. That is tolerable for a positive assertion, because a broken locator makes the test fail. It is dangerous for a negative one, because a broken locator makes the test **pass**. On a food-safety inspection form, `driver.findElements(By.cssSelector(".critical-violation-banner")).isEmpty()` returns `true` when the violation was corrected and the banner went away, and it returns `true` just as cheerfully when the markup was renamed to `.violation-critical` and the banner is sitting on screen in front of you. ## The three ways the check goes green for the wrong reason 1. **The locator can never match.** A rename, a moved container, a frame boundary, a selector that was wrong the day it was written. The check has never once observed the banner and has been passing since it was merged. 2. **The catch is too wide.** `try { driver.findElement(banner); fail(); } catch (Exception e) { }` swallows far more than absence. `InvalidSelectorException` from malformed CSS, a stale reference, a session error — every one of them lands in that catch and is reported as "the banner is gone". 3. **The check ran before the change could happen.** The banner is still on its way out, the list is still rendering, and the empty result is a snapshot of a page that has not finished reacting. What the suite should treat as ready, and how long it waits for it, is a synchronisation decision owned by Selenium's wait mechanisms; the part that belongs here is that the empty list is a fact about one instant, not about the page's final state. ## Writing an absence check that is able to fail The discipline is to make the locator prove itself while the element is present, in the same test: - **Assert presence first.** Before the action that should remove the banner, assert that the same locator matches. Now an empty result afterwards means removal, not a typo. - **Use the plural call, not exception control flow.** `findElements(...).isEmpty()` is a value you assert on; a caught exception is a branch that can be entered for reasons you did not intend. - **If you must catch, catch exactly `NoSuchElementException`,** never `Exception` and never `WebDriverException`, so an invalid selector still fails the test instead of passing it. - **Keep the locator identical** in the presence proof and the absence assertion. Two spellings of "the banner" defeat the whole arrangement. | Check | Banner present | Banner removed | Locator broken | |---|---|---|---| | `findElements(...).isEmpty()` | `false` — correct | `true` — correct | `true` — wrong, silently | | `try findElement / catch Exception` | no throw — correct | throws — correct | throws — wrong, silently | | presence proof, then `isEmpty()` | `false` — correct | `true` — correct | fails at the presence proof | Only the third row can distinguish the three cases, and it does so with the same two calls; the difference is that one of them runs while the element is still supposed to exist. ## Presence in the tree is not visibility The find result reports whether a node matched, nothing more. A banner that is still in the DOM but hidden by CSS is matched and returned exactly like a visible one, so `isEmpty()` is `false` and an absence assertion built on it fails even though nothing is on screen. If the requirement is "the inspector no longer sees a critical-violation warning", presence is the wrong question, and the assertion has to be about the element's displayed state instead. Choose the question deliberately: removal from the document and disappearance from the screen are different product behaviours, and only one of them is what the find result measures. ## What an implicit wait does to a negative check This is Selenium 4, where the implicit wait is set with `driver.manage().timeouts().implicitlyWait(Duration)` and defaults to zero. The specified find algorithm retries **while the match list is empty** until that timer fires, which has an asymmetric effect: - A positive lookup gets faster to write and usually returns as soon as the element appears. - A negative check gets slower on **every** run: nothing will ever match, so the call only answers after the whole timeout has elapsed, and it then returns the same empty list it would have returned immediately. The result is unchanged — an empty list, no exception — but its price is the full timeout, paid by every absence assertion in the suite. How large that timeout should be, and whether an implicit wait belongs in the suite at all, are wait-configuration decisions; what belongs here is knowing that the negative check is the case that pays for it.

  • Why is catching NoSuchElementException around findElement worse than calling findElements?
    Because it turns absence into control flow. The catch has to be exactly right to mean what you intend, and any wider catch reports unrelated failures as absence. The plural call returns an empty list with no exception at all, so the assertion is a value check that reads the same way as every other assertion in the test.
  • What does a non-zero implicit wait do to an absence check?
    It makes it slow without making it better. Element location retries while the match list is empty, so a check on something that will never appear only answers after the whole timeout elapses, on every run, and then returns the same empty list. In Selenium 4 that timeout is set with implicitlyWait(Duration) and defaults to zero.
  • How do you distinguish removed from the document and merely hidden?
    They are different questions. A hidden element still matches a locator and is still returned, so a presence check reports it as present. If the requirement is that the inspector no longer sees the warning, assert the element's displayed state instead; if it is that the markup is gone, the empty match list is the right evidence.

saying these in an interview costs you the question

  • Writes absence as try findElement, catch anything, and call it a pass
  • Catches Exception around a lookup, so an invalid selector reads as absence
  • Never checks that the absence locator matches anything while the element is present
  • Assumes an empty result means removed rather than renamed or moved
  • Thinks an implicit wait makes a negative check faster rather than slower