How do you run only some Catch2 tests from the command line and write JUnit XML for CI in the same run?
answer
- test specs: names, wildcards, tags
- a tilde excludes
- reporters take options after ::
- several reporters in one run
basics
~20 sPass a test spec, such as a name, "area*" or "[geometry]~[slow]", to the test binary, and choose reporters with -r. Since Catch2 3.0.1, several can run at once: -r console -r junit::out=results.xml writes readable output and a JUnit file.
solid answer
~50 sArguments that are not options form a **test spec**. A full name selects one case, a name with `*` at the start or end is a wildcard, `[tag]` selects by tag, adjacent specs must all match, a comma means or, and `~` excludes; hidden cases run only when a spec selects them. `-c "section name"` narrows a run to one section, and `--list-tests` shows what a spec matches. Output format is the **reporter**: `-r junit` writes JUnit-style XML, and options follow `::`, so `-r junit::out=results.xml` writes to a file. Since Catch2 3.0.1, `-r` can be given more than once, so `-r console -r junit::out=results.xml` gives human output and a CI file from one run. The binary exits with 0 when all tests pass and 42 when any fail. Since 3.9.0 the default order is random, so pass `--order decl` or a fixed `--rng-seed` to reproduce an order.
code
bash · 8 lines# What does the spec match?
./tests "[geometry]~[slow]" --list-tests
# Run it, with readable output and a JUnit file for CI
./tests "[geometry]~[slow]" -r console -r junit::out=results.xml
# Only one section of one test case, in declaration order
./tests "generators inside sections" -c "two" --order declgo deeper
Recall how to select tests by name, wildcard and tag, and how -r junit::out=file writes a JUnit report.
Explain how specs combine, why hidden cases need an explicit spec, how several reporters share one run, and what the exit codes signal.
Build CI invocations that are reproducible and fail loudly: fixed seeds for random order, specs that cannot silently match nothing, sharding and per-job reports.
Standardise the test-binary interface across a codebase's suites, so every CI job selects, orders and reports tests the same way and failures reproduce locally.
## The test binary is the runner A Catch2 test executable carries its own command-line interface, whether the `main` comes from `Catch2WithMain` or from your own `Catch::Session`. Everything below works the same way when the binary runs under CTest, because CTest simply passes arguments to it. ## Selecting tests with test specs Positional arguments form a **test spec**. Without one, the binary runs every non-hidden case. The basic specs: - **A full name**, such as `"area is positive"`, selects that case. - **A wildcarded name**, with `*` only at the start or end, such as `"area*"` or `"*triangle*"`. - **A tag**, such as `[geometry]`. They combine: 1. Specs written next to each other must **all** match: `[geometry][fast]`. 2. A **comma** means any: `[geometry],[migration]`. Commas bind more loosely than concatenation, so `[a][b],[c]` means both a and b, or c. 3. A **tilde** rejects: `[geometry]~[slow]`. Rejection wins over everything else, and `~` binds only to the next basic spec. A hidden case, tagged `[.]`, `[.name]` or `[!benchmark]`, runs only when a positive spec selects it. Quote specs in the shell so `*` and brackets reach the binary unchanged, and use `--list-tests` with the same spec to see what it matches before running anything. ## Narrowing further - `-c "section name"` runs only that section path; repeat `-c` for nested sections. - Catch2 3.13.0 added `-g` and `-p` for generator elements and general path filters. - `-#` adds a `[#filename]` tag to every case, so `"[#test_area]"` selects one source file's cases. - `--shard-count` and `--shard-index` split the cases across parallel CI jobs. ## Listing and sharding - `--list-tests` prints the matching cases with their tags, and `--list-tags` prints every tag with a count; both accept the same test spec as a run. - `--shard-count N --shard-index i`, added in Catch2 3.0.1, splits the selected cases evenly into `N` sets and runs set `i`, counting from 0, so `N` parallel CI jobs can each run one share of a large suite. - Sharding happens after filtering, so every job must pass the same spec and the same seed for the shares to cover the suite exactly once. ## Choosing output with reporters The **reporter** formats results. `--list-reporters` printed these for 3.16.1: `Automake`, `compact`, `console` (the default), `JSON`, `JUnit`, `SonarQube`, `TAP`, `TeamCity` and `XML`. | Need | Arguments | |---|---| | Readable local output | default `console`, `-s` to show passing assertions too | | JUnit XML file for CI | `-r junit::out=results.xml` | | Both from one run | `-r console -r junit::out=results.xml` | | Quiet console | `-r console::verbosity=quiet` | Reporter options follow `::` as `key=value` pairs. Running **several reporters** at once was introduced in Catch2 3.0.1; before that, one run produced one report format. Each reporter needs its own `-r`: a comma-joined `-r console,junit` was rejected with `Unrecognized reporter, 'console,junit'`. A run with `-r console -r junit::out=res.xml` printed the console summary and wrote a `<testsuites>` document. How a CI server reads that JUnit file is a separate subject. ## Order, seeds and exit codes - **Order.** Since Catch2 3.9.0 the default `--order` is `rand`; `decl` (declaration order) and `lex` (by name) are the alternatives. A suite that only passes in one order has hidden coupling, which a random default exposes. - **Seed.** The seed is printed at the start of each run; `--rng-seed <n>` reproduces both the order and the random generator values. - **Exit code.** The `Catch::Session` constants in 3.16.1 include `TestFailureExitCode` = 42 and `NoTestsRunExitCode` = 2. A spec that matched nothing printed `No test cases matched` and exited with 2, so a CI job with a typo in its spec fails instead of passing vacuously. ## A typical CI line ```bash ./tests "[geometry]~[slow]" --order rand --rng-seed 1234 \ -r console::verbosity=quiet -r junit::out=results.xml ``` This runs the fast geometry cases in a reproducible random order, keeps the log short, and leaves a JUnit file for the CI server.
- Why might a Catch2 suite start failing after an upgrade to 3.9.0 without any test changes?Catch2 3.9.0 changed the default run order to random. Tests that silently depended on running after another test, through a global, a static cache or a file, now meet a different order and fail intermittently. Rerunning with the printed `--rng-seed` reproduces the order; the fix is removing the shared state, with `--order decl` only as a temporary bridge.
- What exit code does a Catch2 binary return when its test spec matches no test cases?It prints `No test cases matched` and exits with 2, the `NoTestsRunExitCode` constant in `Catch::Session`. That makes a mistyped filter in a CI script fail visibly. A test failure exits with 42 (`TestFailureExitCode`), and `--allow-running-no-tests` exists for jobs where an empty selection is legitimate.
- A space between two tags in a Catch2 spec means either tag.
- Catch2 can only write one report format per run.
- Catch2 runs tests in declaration order unless told otherwise.
- A failing Catch2 run exits with 1.
- Hidden tests run whenever no filter is given.
Assumes Catch2 3: several reporters per run since 3.0.1, random default order since 3.9.0, generator and path filters since 3.13.0; checked against 3.16.1.