What does @unittest.expectedFailure do, and what happens if that test passes?
answer
- Unlike a skip, the body still runs
- The verdict is inverted, not suppressed
- Passing is the interesting case
- It is called an unexpected success
- wasSuccessful returns False since 3.4
basics
~20 sThe @unittest.expectedFailure decorator runs the test and inverts its verdict: a failure or error is recorded as an expected failure and does not break the run. If the test unexpectedly passes, it is recorded as an unexpected success and the run is reported as failed.
solid answer
~40 s`@unittest.expectedFailure` marks a test whose failure is a known, accepted defect. The body **does** execute — unlike a skip — and if it fails or errors, the result goes into the expected-failures bucket and the summary still reads `OK (expected failures=1)`. If it passes, `unittest` records an **unexpected success**, and since Python 3.4 `unittest.TestResult.wasSuccessful()` returns `False` for that, so `python -m unittest` exits non-zero and prints `FAILED (unexpected successes=1)`. That asymmetry is deliberate: it turns the moment somebody accidentally fixes the bug into a build failure that forces you to delete the decorator, instead of letting a stale marker sit there forever pretending a passing test is still broken.
code
python · 16 linesimport unittest
class OrderingTests(unittest.TestCase):
@unittest.expectedFailure
def test_known_ordering_defect(self):
self.assertEqual(["page-1", "page-2"], ["page-2", "page-1"])
@unittest.expectedFailure
def test_defect_already_fixed(self):
self.assertEqual(["page-1"], ["page-1"])
suite = unittest.defaultTestLoader.loadTestsFromTestCase(OrderingTests)
result = unittest.TextTestRunner(verbosity=2).run(suite)
print("successful:", result.wasSuccessful())go deeper
Remember the two words: expected failure when the test fails as predicted, unexpected success when it passes anyway. The key fact to recall is that the test actually runs, unlike a skipped one.
Explain the inverted verdict and the exit status: an expected failure keeps the run green, an unexpected success makes unittest.TestResult.wasSuccessful() return False. Contrast it with a skip, where the body never executes.
Be ready to argue when a defect deserves a pinned reproduction rather than a skip, and to explain why the marker is unsuitable for flaky or process-crashing tests since the body still runs.
Own the accounting: expected failures should map one-to-one onto tracked defects, and a policy for how long one may live. Be ready to say why the red build on an unexpected success is a feature you would not configure away.
### Two different tools for two different situations `@unittest.skip` says *do not run this*. `@unittest.expectedFailure` says *run this, I know it fails, do not break the build over it*. The distinction matters more than it looks: | | skip | expectedFailure | |---|---|---| | body executes | no | **yes** | | costs run time | no | yes | | can crash the process | no | yes | | records the current defect | no | yes, as a live reproduction | | notices when it starts passing | no | **yes, loudly** | An expected failure is a *pinned reproduction of a known bug*. It keeps running against every commit, so the assertion stays in sync with the code around it, and the moment the bug is fixed the suite tells you. ### The four outcomes When the runner executes a test decorated with `@unittest.expectedFailure`: * the body raises an `AssertionError` → recorded as an **expected failure**; the run is still successful. * the body raises any other exception → also recorded as an **expected failure**; `unittest` does not require the failure to be of a particular kind, which is worth knowing because it means an `ImportError` or a typo in the test is swallowed just as happily as the bug you meant to pin. * the body passes → recorded as an **unexpected success**. * the body raises `unittest.SkipTest` (from `self.skipTest`) → recorded as a skip; skipping wins over the expectation. The summary line separates them: ``` Ran 2 tests in 0.001s FAILED (expected failures=1, unexpected successes=1) ``` ### The unexpected-success rule, and its history Whether an unexpected success should fail the build was genuinely contentious. Early Python 3 releases counted it but still reported the run as successful; from **Python 3.4** onward `unittest.TestResult.wasSuccessful()` returns `False` whenever the run recorded any unexpected success, and that is still the behaviour on 3.14. So a test that starts passing turns the build red. Being red is the useful part. Without it, an `expectedFailure` decorator on a now-passing test is pure misinformation: a reader believes the feature is still broken, a triage sweep skips over the ticket, and the assertion silently stops being a signal about anything. Forcing the failure means the fix and the decorator's removal land in the same change, which is the only way the marker stays truthful. Note that `unittest` gives you exactly one behaviour here — there is no non-strict mode where an unexpected success is tolerated. If you need something tolerant, you are reaching for a skip, and should say so with a reason. ### When to use it Use `expectedFailure` when: * a bug is confirmed, reproducible and written down, and the fix is scheduled but not now; * you are writing a test *first*, against behaviour that does not exist yet, and want it in the tree without breaking the build; * you want the suite to tell you when an upstream fix reaches you. Do **not** use it when: * the test is *flaky* rather than failing. A flaky test decorated with `expectedFailure` will eventually pass and turn the build red for the wrong reason — flakiness needs a skip with an owner, or a fix. * the test crashes or hangs the process. The body still runs, so a segfault or an infinite loop is not contained by the decorator; that case needs a skip. * you simply want the number of red tests to go down. Both markers are debt; the only difference is which one is honest about what is happening. ### Reading the count In review and in triage, treat the two buckets differently. Skips answer "what are we not running?" Expected failures answer "what do we know is broken?" A suite with a rising expected-failure count is not decaying in the same way a suite with a rising skip count is — it still has live, executing reproductions of every one of those bugs. What it does need is that the number correspond one-to-one with tracked defects, so that "expected failures=7" and "seven open bugs" are the same seven.
- Does @unittest.expectedFailure care which exception the test raises?No. Any exception out of the body — an `AssertionError`, an `ImportError`, a `TypeError` from a typo in the test itself — counts as the expected failure. That is a real weakness: the marker pins *that the test fails*, not *why*. If the reason matters, assert on it explicitly inside the body instead of relying on the decorator alone.
- Why is an unexpected success treated as a build failure rather than a pass?Because the decorator is now lying. Somebody fixed the defect and the test still advertises it as broken, so triage and readers are misled. Failing the run forces the decorator's removal into the same change as the fix, which keeps the marker honest. `unittest.TestResult.wasSuccessful()` returns `False` whenever any unexpected success is recorded.
- Would you use expectedFailure for a test that fails intermittently?No. An intermittent test will sometimes pass, which turns the build red as an unexpected success on a random run — the worst of both worlds. Intermittency calls for a skip with a named owner and reason while the cause is investigated, or for fixing the source of the nondeterminism.
A skip is taking a broken machine off the production line; an expected failure is leaving it running with a red light on it, so the day it stops failing, somebody notices.
saying these in an interview costs you the question
- Thinks expectedFailure prevents the test body from running
- Says an unexpected success is reported as a pass
- Believes the run stays green when the test starts passing
- Treats expectedFailure as a fix for a flaky test
- Assumes only AssertionError counts as the expected failure