skip to content

Skips and Expected Failures

Marking a test skipped or known-failing instead of commenting it out — skip decorators, runtime skipTest, expectedFailure. Interviewers ask what a pile of permanently skipped tests really says.

part ofPythonoverview, primer and where to startread it →
on this pageshow

questions

4

What does @unittest.skip do to a test, and why prefer it to commenting the test out?

level: juniorimportance: must knowfreq 40%

answer

  1. The runner needs to know it exists
  2. A distinct outcome, not a pass
  3. Reason string survives; a comment does not
  4. Counted in the summary line
  5. skip, skipIf, skipUnless; method or class

basics

~20 s

The @unittest.skip decorator stops the decorated test body from running and makes the runner report it as skipped, with the reason you supplied. The test is still collected and still counted, so the gap stays visible in the summary; a commented-out test disappears entirely.

solid answer

~40 s

`@unittest.skip(reason)` wraps a test method (or a whole `unittest.TestCase` subclass) so the body never executes and the runner records a **skip**, not a pass. Skips are a distinct outcome: the summary line reads `OK (skipped=3)`, and running with `-v` prints each skipped test next to its reason. `@unittest.skipIf(condition, reason)` and `@unittest.skipUnless(condition, reason)` do the same thing conditionally, typically on a platform or capability check. The reason it beats commenting out is accounting: a commented test is invisible to the runner, to coverage tooling and to reviewers, so nobody ever notices that it stopped protecting anything. A skip leaves a named, counted, greppable record of exactly what is not running and why — which is the only thing that makes the gap fixable later.

code

python · 16 lines
python
import sys
import unittest


@unittest.skipIf(sys.platform == "win32", "converter shells out to a POSIX-only tool")
class ConverterTests(unittest.TestCase):
    @unittest.skip("blocked on the batch-ordering fix")
    def test_pages_keep_input_order(self):
        self.fail("body never runs")

    @unittest.skipUnless(sys.version_info >= (3, 12), "needs 3.12 or newer")
    def test_modern_path(self):
        self.assertTrue(True)


unittest.main(argv=["prog"], verbosity=2, exit=False)

go deeper

for a junior

Be ready to state the three decorators — @unittest.skip, @unittest.skipIf, @unittest.skipUnless — and to say that a skipped test does not run and is reported in its own bucket with the reason you gave.

for a middle

Explain the mechanics: the decorator raises unittest.SkipTest before the body, it can be applied to a whole unittest.TestCase subclass, and the reason surfaces in verbose output. Say why the runner still counting the test is the actual benefit.

for a senior

An interviewer expects you to talk about skips as debt you operate: reasons that name something actionable, a tracked skip count, and the difference between a permanent platform exclusion and a quarantined failure nobody owns.

for a principal

Own the policy angle — who is allowed to add a skip, what makes a suite's skip count a release signal rather than noise, and when a permanently skipped area should simply be deleted along with the feature it no longer protects.

### Skip is an outcome, not an absence A `unittest` run classifies every collected test into one of several outcomes: pass, failure, error, skip, expected failure, unexpected success. `@unittest.skip` moves a test into the *skip* bucket. The decorator replaces the method with a wrapper that raises `unittest.SkipTest` immediately, and `unittest.TestCase` catches that exception specially: the test's body never runs, `setUp` is not even entered for a method-level skip, and the result object records the test together with the reason string rather than a traceback. That distinction is the whole point. The runner's final line separates the buckets: ``` Ran 12 tests in 0.004s OK (skipped=3) ``` and in verbose mode each one is attributed: ``` test_pages_keep_input_order (tests.ConverterTests) ... skipped 'blocked on the batch-ordering fix' ``` ### The three decorators * `@unittest.skip(reason)` — always skip. Since Python 3.11 it may also be applied bare, as `@unittest.skip`, without a reason; supplying one is still much better practice. * `@unittest.skipIf(condition, reason)` — skip when `condition` is truthy. * `@unittest.skipUnless(condition, reason)` — skip unless `condition` is truthy. All three work on a single test method **and** on a `unittest.TestCase` subclass. Applied to the class, every test method inside it is reported skipped — which is how one line silently removes a whole file's worth of coverage, and why class-level skips deserve more scrutiny than method-level ones. A fourth route is runtime rather than decoration: `self.skipTest(reason)` inside a test body, or raising `unittest.SkipTest` directly, aborts the test at that point and records the same skip outcome. ### Why not just comment it out Commenting out a failing test — or renaming `test_foo` to `_test_foo`, which is the same move in disguise — makes the test vanish from collection. Concretely: 1. **The count lies.** "Ran 118 tests, OK" reads identically whether the 119th was deleted yesterday or never existed. A skip keeps the total honest and puts a number on the debt. 2. **The reason is lost.** A commented block records no cause. Six months later nobody knows whether it was a flaky environment, a deliberate platform exclusion, or a real product bug somebody chose not to fix. 3. **Tooling stops seeing it.** Linters and static analysis skip commented code; a skipped test is still parsed, still imported, still type-checked, and still breaks loudly when the API it calls is renamed. That last property matters most: a commented test rots into something that no longer even compiles, so re-enabling it becomes an archaeology project rather than deleting one decorator line. 4. **It is not reviewable.** A skip decorator with a reason string shows up in a diff as an explicit, arguable decision. A commented-out block reads as noise and gets waved through. ### The counter-pressure Skips are cheap to add and nobody is paged when the count grows, so a suite drifts toward a permanent pile of them. The honest discipline is to treat the skip count as a tracked number with an owner and a reason that names something actionable — a ticket, a platform, a missing capability — and never as a way to make a red build green. A reason like `"flaky"` is the smell; `"skipped on Windows: the converter shells out to a POSIX-only tool"` is a decision. Two skips are also genuinely permanent and healthy: platform exclusions (`sys.platform` checks) and capability exclusions (an optional interpreter feature or codec is not present in this build). Those should never be deleted — they are the test suite correctly describing where it applies. ### What it is not A skip is not a pass and not a failure. `unittest.TestResult.wasSuccessful()` returns `True` with skips present, so a skipped suite exits zero — which is exactly why a growing skip count needs a deliberate check of its own rather than relying on the build going red.

  • Does a skipped test make the run unsuccessful?
    No. Skips are reported separately and `unittest.TestResult.wasSuccessful()` still returns `True`, so `python -m unittest` exits zero with any number of skips. That is convenient and dangerous in equal measure: coverage can fall to nothing without the build ever going red, so a team that relies on skips needs a separate check on the skip count or on the specific tests allowed to skip.
  • What happens if you put @unittest.skip on the TestCase subclass instead of one method?
    Every test method in that class is reported skipped with the same reason, and the class's own setup never runs. It is the fastest way to disable a file, which is precisely why it deserves review: one line can remove dozens of tests while the summary still says OK.
  • How do you skip a test only when an optional capability is missing?
    Compute the probe once at module level — for example checking whether a module can be found, or testing a platform attribute — and pass the boolean to `@unittest.skipUnless(available, reason)`. Keep the probe cheap, because it runs at import time on every run, including runs where that test is filtered out.

Commenting out a test is boarding up a window; skipping it is hanging an OUT OF ORDER sign with a date and a name on it. Both stop the draught, but only one tells the next person what to fix.

saying these in an interview costs you the question

  • Says a skipped test counts as a passing test
  • Claims commenting out a test is equivalent to skipping it
  • Thinks @unittest.skip only works on methods, not classes
  • Believes a skip makes the run exit non-zero
  • Gives no reason string, or a reason like 'flaky'
  • Thinks the skipped body still executes but is ignored

context

open as a page

What does @unittest.expectedFailure do, and what happens if that test passes?

level: middleimportance: should knowfreq 22%

basics

~20 s

The @unittest.expectedFailure decorator runs the test and inverts its verdict: a failure or error is recorded as an expected failure and does not break the run. If the test unexpectedly passes, it is recorded as an unexpected success and the run is reported as failed.

open as a page

When is @unittest.skipIf's condition evaluated, and when must you call self.skipTest instead?

level: middleimportance: should knowfreq 30%

basics

~20 s

A skipIf condition is an ordinary expression evaluated once, when the decorator line executes at import time — before any test or its setup runs. Use self.skipTest(reason) inside setUp or the test body whenever the decision depends on state that only exists at run time.

open as a page

A unittest suite reports 40 skips on CI and none locally. How do you find what silently stopped running?

level: seniorimportance: should knowfreq 28%

basics

~20 s

Run the suite verbosely so every skip prints next to its reason, then trace each reason back to the condition that produced it. Skips are almost always import-time environment checks, so compare the CI image's platform, environment variables and optional capabilities against the local machine.

open as a page