When do doctests earn their place, and when should a dedicated unit test replace them?

seniorshould knowfreq 26%

answer

The reader is the first audience
Documentation that cannot silently drift
No parametrization, almost no fixtures
Flags multiplying is the smell
Illustrative example, assertion in a test

basics

~20 s

Doctests earn their place when the example is documentation first: a short, pure, deterministic call whose printed form is stable. Anything needing tolerance, fixtures, patching or many input cases belongs in a dedicated test instead.

solid answer

~50 s

The unique value of a doctest is that its failure means the documentation is wrong — so use it where a human reader benefits from the example: library API usage, a tutorial file run through doctest.testfile, an edge case explained beside the code. Move out anything that needs numeric tolerance, unittest.mock.patch, a fixture, IO, or a dozen parametrized cases, because doctest has no parametrization, almost no setup, and couples you to exact printed output. Concretely: when a four-person team's ad-auction bidder starts failing nightly on floating-point rounding drift, the answer is not to edit the expected value or to bury it under ELLIPSIS — it is to keep one rounded, readable example in the docstring and assert the real number in a test with math.isclose and an explicit tolerance. Then run the doctests in the same CI job as everything else, or they rot.

The unique value of a doctest is that its failure means the **documentation** is wrong — so use it where a human reader benefits from the example: library API usage, a tutorial file run through `doctest.testfile`, an edge case explained beside the code. Move out anything that needs numeric tolerance, `unittest.mock.patch`, a fixture, IO, or a dozen parametrized cases, because doctest has no parametrization, almost no setup, and couples you to exact printed output. Concretely: when a four-person team's ad-auction bidder starts failing nightly on floating-point rounding drift, the answer is not to edit the expected value or to bury it under `ELLIPSIS` — it is to keep one rounded, readable example in the docstring and assert the real number in a test with `math.isclose` and an explicit tolerance. Then run the doctests in the same CI job as everything else, or they rot.

code

python · 22 lines

python

import doctest
import math
import unittest


def clearing_price(bids):
    """The second-highest bid wins the slot.

    >>> round(clearing_price([0.10, 0.30, 0.20]), 2)
    0.2
    """
    return sorted(bids)[-2]


class ClearingPriceTest(unittest.TestCase):
    def test_within_tolerance(self):
        self.assertTrue(math.isclose(clearing_price([0.10, 0.30, 0.20]), 0.2))


if __name__ == "__main__":
    print(doctest.testmod())
    unittest.main()

go deeper

for a junior

Recall the basic split: a docstring example is there to teach a reader, a test is there to catch a regression. If the example only exists to make the build check something, it probably belongs in a test file.

for a middle

Explain what doctest structurally cannot do — parametrization, fixtures, mocking, tolerance — and why exact-output matching is both the feature and the cost. Be able to restructure a failing example instead of patching its expected text.

for a senior

Show production judgement on a real failure: diagnose why the example broke, refuse the cosmetic fixes, split documentation from assertion, and make sure the examples run in the same CI job as everything else so they cannot rot.

for a principal

Own the policy across teams: where verified examples are required, which flags need justification, and how you keep a documentation tool from quietly becoming a second test suite with worse diagnostics and no ownership.

What doctest is uniquely good at

A doctest is the only kind of test whose failure means the documentation is wrong. That is the whole case for it, and it is a real one. Where it fits:

Small, pure, deterministic functions whose call and result read well in a docstring.
Library API examples — the block a reader will copy into a REPL, kept honest automatically.
Tutorial and README prose, run with doctest.testfile or doctest.DocFileSuite, so a narrative guide cannot rot into fiction.
Explaining an edge case in place, next to the code that implements it, where a reader looking at the function will actually see it.

What it is structurally bad at

Exact-output coupling. The assertion is a string comparison, so every float, set, address, path and timestamp is a false-failure source.
No parametrization. Twelve input cases means twelve lines of transcript in a docstring nobody will read as documentation any more. The docstring has stopped being documentation and become a table.
Almost no fixtures. You get globs and extraglobs, plus setUp/tearDown on DocFileSuite. Anything needing a database, a temporary tree, or unittest.mock.patch does not belong in a docstring.
Order coupling and cascading failures. All examples in one docstring share a namespace and run in order; an early failure leaves later names unbound, so one defect prints as several failures.
Weak diagnostics. A wall of expected-vs-got text, no structured assertion message. REPORT_NDIFF helps and does not fix it.
Pressure on the design. Once a docstring is the test, there is a pull to make functions print, or return prettily-repr-able values, for the test's convenience. That is the tail wagging the dog.

A worked judgement call

A four-person team owns an ad-auction bidder. Its clearing_price docstring shows the second-highest bid winning the slot, and the nightly run has started failing on a floating-point rounding drift after a change to how bids are accumulated: the example says 0.3 and the code now prints 0.30000000000000004. Three responses, in ascending order of quality:

Edit the expected output to the drifted value. The suite goes green and now pins a representation artefact as documented behaviour. The next re-association of the same additions breaks it again.
Add flags until it passes. ELLIPSIS on that line makes the example assert nearly nothing, and the documentation now shows the reader an ellipsis where a price should be.
Split the concerns. Keep one illustrative example in the docstring that prints something a human wants to read — round(clearing_price(bids), 2) → 0.2 — and move the real numeric assertion into a dedicated test using math.isclose with an explicit tolerance, where the tolerance is visible, named and reviewable. The docstring stays documentation; the test stays a test.

The general form of that move: ask what the example is for. If a reader benefits from seeing it, it is documentation and doctest should keep it honest. If only the build benefits, it is a test and belongs in a test.

Operating them

Run them where the rest of the suite runs. Doctests that no runner collects rot within a release. Wire them in through load_tests with doctest.DocTestSuite, or run python -m doctest as a CI step and gate on its non-zero exit.
Keep them fast and offline. No network, no clock, no filesystem. A doctest that needs a fixture has already told you it is a unit test.
Use # doctest: +SKIP deliberately. For a line that is genuinely illustrative — a call that needs credentials, or a long-running job — skipping is honest. Skipping to silence a real failure is not, and it should be visible in review.
Set a house rule on flags. For a four-person team the cheap version is: NORMALIZE_WHITESPACE run-wide is fine; ELLIPSIS needs a reason in the diff; more than one flag on an example is a review comment.

The one-sentence position

doctest is a documentation tool with a test harness attached, not a test framework with documentation attached — use it where the example's first audience is a human reader, and reach for a dedicated test the moment the example exists only to satisfy the build.

### What doctest is uniquely good at A doctest is the only kind of test whose failure means *the documentation is wrong*. That is the whole case for it, and it is a real one. Where it fits: * **Small, pure, deterministic functions** whose call and result read well in a docstring. * **Library API examples** — the block a reader will copy into a REPL, kept honest automatically. * **Tutorial and README prose**, run with `doctest.testfile` or `doctest.DocFileSuite`, so a narrative guide cannot rot into fiction. * **Explaining an edge case in place**, next to the code that implements it, where a reader looking at the function will actually see it. ### What it is structurally bad at * **Exact-output coupling.** The assertion is a string comparison, so every float, set, address, path and timestamp is a false-failure source. * **No parametrization.** Twelve input cases means twelve lines of transcript in a docstring nobody will read as documentation any more. The docstring has stopped being documentation and become a table. * **Almost no fixtures.** You get `globs` and `extraglobs`, plus `setUp`/`tearDown` on `DocFileSuite`. Anything needing a database, a temporary tree, or `unittest.mock.patch` does not belong in a docstring. * **Order coupling and cascading failures.** All examples in one docstring share a namespace and run in order; an early failure leaves later names unbound, so one defect prints as several failures. * **Weak diagnostics.** A wall of expected-vs-got text, no structured assertion message. `REPORT_NDIFF` helps and does not fix it. * **Pressure on the design.** Once a docstring is the test, there is a pull to make functions print, or return prettily-repr-able values, for the test's convenience. That is the tail wagging the dog. ### A worked judgement call A four-person team owns an ad-auction bidder. Its `clearing_price` docstring shows the second-highest bid winning the slot, and the nightly run has started failing on a floating-point rounding drift after a change to how bids are accumulated: the example says `0.3` and the code now prints `0.30000000000000004`. Three responses, in ascending order of quality: 1. **Edit the expected output to the drifted value.** The suite goes green and now pins a representation artefact as documented behaviour. The next re-association of the same additions breaks it again. 2. **Add flags until it passes.** `ELLIPSIS` on that line makes the example assert nearly nothing, and the documentation now shows the reader an ellipsis where a price should be. 3. **Split the concerns.** Keep one illustrative example in the docstring that prints something a human wants to read — `round(clearing_price(bids), 2)` → `0.2` — and move the real numeric assertion into a dedicated test using `math.isclose` with an explicit tolerance, where the tolerance is visible, named and reviewable. The docstring stays documentation; the test stays a test. The general form of that move: **ask what the example is for.** If a reader benefits from seeing it, it is documentation and doctest should keep it honest. If only the build benefits, it is a test and belongs in a test. ### Operating them * **Run them where the rest of the suite runs.** Doctests that no runner collects rot within a release. Wire them in through `load_tests` with `doctest.DocTestSuite`, or run `python -m doctest` as a CI step and gate on its non-zero exit. * **Keep them fast and offline.** No network, no clock, no filesystem. A doctest that needs a fixture has already told you it is a unit test. * **Use `# doctest: +SKIP` deliberately.** For a line that is genuinely illustrative — a call that needs credentials, or a long-running job — skipping is honest. Skipping to silence a real failure is not, and it should be visible in review. * **Set a house rule on flags.** For a four-person team the cheap version is: `NORMALIZE_WHITESPACE` run-wide is fine; `ELLIPSIS` needs a reason in the diff; more than one flag on an example is a review comment. ### The one-sentence position doctest is a documentation tool with a test harness attached, not a test framework with documentation attached — use it where the example's first audience is a human reader, and reach for a dedicated test the moment the example exists only to satisfy the build.

A docstring example needs credentials and a network call. What do you do with it?
Take it out of the doctest path. Keep a short offline example in the docstring showing the call shape, mark the live line `# doctest: +SKIP` if it must stay visible, and write the real test separately with `unittest.mock.patch` standing in for the remote call. Doctests should be fast, offline and deterministic or they will be disabled by whoever is on call.
How do you stop doctests rotting once they exist?
Give them exactly one runner in CI. Either add `doctest.DocTestSuite()` through a `load_tests` function so ordinary discovery collects them, or run `python -m doctest` as a build step and gate on its non-zero exit. Uncollected examples diverge from the code within a release and then teach readers something false.
A docstring has grown twelve examples covering edge cases. What is wrong with that?
It has stopped being documentation. Nobody reads a twelve-case transcript to learn the API, and doctest gives you none of the tooling a table of cases deserves — no parametrization, no per-case naming, and a shared namespace where one failure cascades. Keep one or two illustrative examples and move the matrix into a dedicated test.

A doctest is a photograph in a manual that is re-shot on every build: worth keeping while a reader wants to look at it, not worth staging a whole scene for when only the build cares.

saying these in an interview costs you the question

Uses doctest as the project's only test framework
Adds option flags until a fragile example passes
Writes long transcripts nobody reads as documentation
Leaves doctests out of CI so they silently rot
Tests error paths and edge cases only in docstrings
Makes production code print purely to satisfy an example

When do doctests earn their place, and when should a dedicated unit test replace them?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

What doctest is uniquely good at

What it is structurally bad at

A worked judgement call

Operating them

The one-sentence position

▾common follow-ups3 questions

▾analogy~20 s

▾red flags6 flags

§ context

▾related questions4 questions

▾related topics4 topics

What doctest is uniquely good at

What it is structurally bad at

A worked judgement call

Operating them

The one-sentence position

answer

go deeper

context