How do you make a CI run fail on DeprecationWarning without breaking on third-party ones?

### The goal, stated precisely Making warnings fatal means changing one filter action from `ignore` (or `default`) to `error`. When a matching filter says `error`, the warnings machinery raises the warning instance instead of printing it, so the call site blows up with a traceback and the process exits non-zero. That is the entire mechanism behind "fail the build on deprecations" — there is no separate mode, only the filter list. ### The three ways to say it * `python -W error::DeprecationWarning -m unittest discover` * `PYTHONWARNINGS=error::DeprecationWarning python -m unittest discover` * `warnings.simplefilter("error", DeprecationWarning)` in code, or the finer-grained `warnings.filterwarnings("error", category=DeprecationWarning, module=r"myapp(\.|$)")` The `-W` and `PYTHONWARNINGS` spec grammar is identical: `action:message:category:module:lineno`, with empty fields meaning "match anything", so `error::DeprecationWarning` means "error, any message, this category, any module, any line". Two operational differences are worth knowing. `PYTHONWARNINGS` is an environment variable, so it is inherited by every child process the run spawns — a subprocess, a worker process — whereas `-W` applies only to the interpreter you typed it on. And in the `-W`/`PYTHONWARNINGS` form the `module` field is regex-escaped before use, so it is a literal prefix match; only `warnings.filterwarnings(module=...)` accepts a real regular expression. ### Precedence: the last `-W` wins Each `-W` option is inserted at the **head** of the filter list as it is processed, and matching stops at the first hit. So the options are effectively evaluated right-to-left: write the broad rule first and the narrow override last. ``` python -W ignore::DeprecationWarning -W error::DeprecationWarning:myapp -m unittest discover ``` That reads as: fail on deprecations attributed to `myapp` (and `myapp.anything`, since the match is anchored at the start), ignore everybody else's. The same shape in code is a `simplefilter("ignore", DeprecationWarning)` followed by a narrower `filterwarnings("error", ...)`, because `filterwarnings` also inserts at the head unless you pass `append=True`. ### Why the naive `-W error` fails immediately Turn `error` on globally and the first thing that breaks is usually not your code. Import-time deprecations inside dependencies become `ImportError`-shaped crashes, and any library that warns about its own future changes takes the suite down. Worse, the failure is not stable: a dependency upgrade can add a new deprecation and turn a green suite red with no change of yours. That is why the useful configuration is *scoped*, not global. ### The attribution trap Scoping by module is where candidates get burned, and it is the interesting half of this question. The `module` field is matched against the module the warning is **attributed to**, which `stacklevel` chooses — normally the *caller*. A well-behaved library uses `stacklevel=2` precisely so that its deprecation points at your line of code, which means: * Deprecations triggered by your calls into a dependency are attributed to **your** module, so they are caught by a `myapp`-scoped `error` filter. That is what you want. * Deprecations a dependency raises internally, or at import time, are attributed to that dependency, so a `myapp`-scoped filter ignores them. Also what you want. * A library that forgets `stacklevel` will be attributed to itself, and its warnings will slip through your scoped filter even though it is *your* call that provokes them. So a module-scoped `error` filter is a good approximation of "our own technical debt", not a guarantee. When one specific third-party deprecation is unfixable and keeps failing the run, pin it by message regex rather than widening the category: `warnings.filterwarnings("ignore", message=r"the exact text.*", category=DeprecationWarning)` — a narrow, dated, reviewable exception, ideally with a ticket reference in a comment. ### Where to put it, and what else to do with it Put the policy where the test command lives so it applies to every run, and keep the same setting out of production: `error` in a live service converts a cosmetic deprecation into an outage. A common split is `error::DeprecationWarning` scoped to first-party code in CI, `default::DeprecationWarning` under `-X dev` locally, and stock filters in production with the warnings routed into the log stream by `logging.captureWarnings(True)` so they are still countable. Two refinements worth naming. First, tests that deliberately exercise a deprecated path should assert on the warning rather than fight the filter — `warnings.catch_warnings()` gives you a scoped filter list that is restored on exit. Second, `error` on `DeprecationWarning` alone is the safe start; `FutureWarning` and `PendingDeprecationWarning` are separate categories and are not covered by it, and `-W error` with no category makes *every* warning fatal, which almost nobody actually wants.