skip to content

How do you run a module's doctests with doctest.testmod, python -m doctest or unittest?

level: middleimportance: should knowfreq 32%

answer

  1. Nothing runs doctests automatically
  2. testmod tests the calling module
  3. The CLI exits non-zero on failure
  4. load_tests folds them into unittest
  5. DocTestSuite for modules, DocFileSuite for files

basics

~20 s

Call doctest.testmod() inside the module to run its own docstring examples, run python -m doctest on a .py or text file from the shell, or add doctest.DocTestSuite() in a load_tests function so a normal unittest run reports them.

solid answer

~40 s

Three routes. `doctest.testmod()` with no argument tests `__main__` — the module it is called from — prints a failure report and returns a `TestResults` tuple of failed, attempted and skipped counts; keep the call under an `if __name__ == "__main__":` guard so importing the module does not run the suite. From the shell, `python -m doctest bidder.py` imports the file and runs its docstring examples, `-v` is verbose, `-o ELLIPSIS` sets an option flag run-wide, `-f` fails fast, and the process **exits non-zero on failure**, which is what a CI step gates on. For one report across the whole suite, `doctest.DocTestSuite()` and `doctest.DocFileSuite()` return unittest suites; adding them in a `load_tests` function means ordinary test discovery collects the examples alongside the real test cases. `doctest.testfile()` covers README-style prose.

code

python · 9 lines
python
import doctest
import tempfile
from pathlib import Path

if __name__ == "__main__":
    with tempfile.TemporaryDirectory() as tmp:
        doc = Path(tmp, "bidder.txt")
        doc.write_text(">>> 2 + 2\n4\n", encoding="utf-8")
        print(doctest.testfile(str(doc), module_relative=False))

go deeper

for a junior

Remember that writing a >>> example does not make it run. Know one way to execute them — doctest.testmod() under a main guard, or python -m doctest thefile.py — and that a passing run prints almost nothing.

for a middle

Explain all three entry points and what each returns or exits with: the TestResults tuple, the non-zero exit status of the CLI, and the unittest suites returned by DocTestSuite and DocFileSuite. Know why the main guard matters.

for a senior

Show how you wire doctests into an existing pipeline so they cannot rot: one runner, one report, a build that fails on exit status rather than on a grepped log, and a clear answer for where option flags are configured for the whole run.

for a principal

Own the standard: whether docstring examples are part of the definition of done, which single runner reports them, and how you avoid a second parallel test mechanism that teams forget to run and reviewers stop trusting.

### Three entry points, one parser The examples themselves are found the same way whichever route you take; what differs is who runs them and who reports the result. ### 1. From inside the module: `doctest.testmod` `doctest.testmod()` with no argument tests the module named `__main__`. It walks that module's docstring and the docstrings of the functions, classes and methods defined in it, runs every example, prints a report of the failures, and returns a `TestResults` tuple carrying `failed`, `attempted` and `skipped` counts. `verbose=True` (or `-v`) prints every example, passing or not. `raise_on_error=True` turns the first failure into an exception instead of a report, which is useful under a debugger. The call belongs under an `if __name__ == "__main__":` guard, and this is not merely style. Without the guard, *importing* the module runs the whole doctest suite as a side effect — including when a test runner or the doctest CLI imports it. There is a sharper version of the same trap: under `python -m doctest yourfile.py`, `__main__` is the `doctest` module itself, so an unguarded `testmod()` at import time tests **doctest's own docstrings** and prints a baffling report. ### 2. From the shell: `python -m doctest` `python -m doctest bidder.py` imports the file and runs the doctests in its docstrings; `python -m doctest guide.txt` treats the file as prose containing `>>>` blocks. The flags are: * `-v` — verbose, report every example; * `-o NAME` — set an option flag such as `ELLIPSIS` or `NORMALIZE_WHITESPACE` for the whole run; repeatable; * `-f` — fail fast, shorthand for `-o FAIL_FAST`. The process **exits non-zero when an example fails**, which is what makes it usable as a CI step. Do not gate a build on grepping the printed report; gate it on the exit status. ### 3. Through `unittest`: `DocTestSuite` and `DocFileSuite` `doctest.DocTestSuite(module)` returns a `unittest.TestSuite` whose test cases are the module's docstring examples; `doctest.DocFileSuite(path, ...)` does the same for text files and additionally accepts `setUp`, `tearDown` and `globs`. The idiomatic wiring is the `load_tests` protocol: define `load_tests(loader, tests, ignore)` in a module, add the doctest suite to `tests`, and any normal unittest discovery run picks the examples up alongside the ordinary test cases. That gives you one runner, one report and one exit status for the whole suite, rather than a second, separately-invoked mechanism nobody remembers to run. `doctest.set_unittest_reportflags()` sets the reporting flags used when doctests run under unittest, since the `-o` command-line route is not available there. ### Where option flags live on each route This is the detail that catches people wiring doctests into a project for the first time. A directive comment on an example (`# doctest: +ELLIPSIS`) works on every route, because it is part of the example's own text. Run-wide flags do not travel: on the API route they are the `optionflags` argument to `testmod`, `testfile`, `DocTestSuite` or `DocFileSuite`; on the CLI they are `-o NAME`, repeatable; under unittest the reporting subset comes from `set_unittest_reportflags`. A flag set on one route has no effect on another, so a suite that passes under `python -m doctest -o ELLIPSIS` will fail the moment discovery picks the same examples up through `load_tests` — unless the flag is also passed to `DocTestSuite`. Prefer per-example directives precisely because they survive the move. ### Wiring it into a build The cheapest durable arrangement is one runner. Put `load_tests` in the modules whose docstrings carry examples, add `DocFileSuite` for the tutorial files, and let the existing test command collect everything: one exit status, one report, no separate step for anyone to forget. Where that is not possible, a `python -m doctest` step is fine as long as the build gates on its exit status and the file list is generated rather than hand-maintained — a hand-maintained list is how examples quietly stop being run. ### Choosing between them * A small script or teaching module: `testmod()` under a `__main__` guard is enough. * A library with a README or tutorial full of examples: `testfile` / `DocFileSuite`, so the prose is verified. * A real project: wire the doctests into whatever already runs in CI. Third-party runners can also collect docstring examples; the stdlib routes above are the ones you can rely on with no dependency at all. ### The failure modes to name * **Doctests that exist but never run.** The most common outcome by far: a module grows examples, nothing collects them, and they rot. Wiring them into the existing runner is the fix. * **Tests that run on import.** An unguarded `testmod()` costs every importer the suite. * **Two reports, one build.** If doctests run in their own step and the unit tests in another, a failure in the first is easy to lose. `load_tests` collapses them into one. * **`testmod` given a path.** It takes a *module object*, not a filename; the filename route is `testfile` or the CLI.

  • Why should the doctest.testmod() call sit under an `if __name__ == "__main__":` guard?
    Otherwise every import of the module runs its whole doctest suite as a side effect. There is a sharper version of the trap: under `python -m doctest yourfile.py` the file is imported while `__main__` is the doctest module itself, so an unguarded `testmod()` tests doctest's own docstrings and prints a report that has nothing to do with your code.
  • How would you make a failing doctest break a CI build?
    Either run `python -m doctest` as a build step and let its non-zero exit status fail the job, or wire the examples into the existing unittest run with `load_tests` and `doctest.DocTestSuite` so one runner and one exit status covers everything. Never gate on parsing the printed report.
  • How do you run the examples in a README-style text file rather than in docstrings?
    `doctest.testfile("guide.txt")` runs the `>>>` blocks in a prose file; paths are resolved relative to the calling module unless you pass `module_relative=False`. `doctest.DocFileSuite` is the unittest equivalent and additionally accepts `setUp`, `tearDown` and `globs`. From the shell, `python -m doctest guide.txt` does the same.

saying these in an interview costs you the question

  • Thinks doctests run automatically when a module is imported
  • Calls testmod() at import time, so importing runs tests
  • Passes a file path to doctest.testmod instead of a module
  • Expects a failing doctest to raise an exception by default
  • Cannot name any way to run doctests in CI
  • Assumes unittest discovery finds docstring examples unaided

context

resolving links…