In CMake, what do enable_testing() and add_test() do, and how does ctest find and run the tests they register?
answer
- CTestTestfile.cmake in the build tree
- top-level directory matters
- exit code zero passes
- target name becomes a path
- include(CTest) adds BUILD_TESTING
basics
~20 senable_testing() makes CMake write CTestTestfile.cmake files into the build tree, and add_test(NAME ... COMMAND ...) records a command in them. ctest, run in the top-level build directory, reads those files and runs each command; exit code 0 passes.
solid answer
~40 s`enable_testing()` turns on test generation for the current directory and below, and belongs in the **top-level** `CMakeLists.txt`, because `ctest` expects a `CTestTestfile.cmake` in the top-level build directory. `add_test(NAME parser_test COMMAND parser_test smoke)` records one test; when the command names an executable target, CMake substitutes the built file's path, and the test runs in `CMAKE_CURRENT_BINARY_DIR` unless `WORKING_DIRECTORY` says otherwise. After building, `ctest --test-dir build` runs the commands: exit code 0 is a pass, anything else a failure, and output is captured. With a multi-config generator you add `-C Release`, or tests report "Not Run". `include(CTest)` calls `enable_testing()` for you unless the `BUILD_TESTING` option it creates is `OFF`, and adds dashboard support such as a default per-test timeout of 1500 seconds.
code
cmake · 10 linescmake_minimum_required(VERSION 3.25)
project(suite CXX)
include(CTest)
add_executable(sleeper sleeper.cpp)
add_test(NAME slow COMMAND sleeper 4)
add_test(NAME fails COMMAND sleeper 0 3)
add_test(NAME quick COMMAND sleeper 0)
set_tests_properties(quick PROPERTIES LABELS "unit;fast")
set_tests_properties(slow PROPERTIES LABELS "integration")go deeper
Call enable_testing() at the top level, register commands with add_test(NAME ... COMMAND ...), build, then run ctest in the build directory.
Explain CTestTestfile.cmake, target-name substitution, the default working directory, -C for multi-config trees, and what include(CTest) adds.
Guard a library's tests so they do not leak into a parent build, and know that include(CTest) brings a 1500-second default timeout.
Decide whether the team needs CDash dashboards or just ctest in CI, and standardise one registration pattern across repositories.
## Three pieces Testing with CMake has three parts: 1. **`enable_testing()`**: tells CMake to generate test files for this directory and below. 2. **`add_test()`**: registers one test, a command line with a name. 3. **`ctest`**: the test driver that ships with CMake; it reads the generated files and runs the commands. Nothing about the test framework matters to CTest. A test is any command whose **exit code** says pass (0) or fail (non-zero), so a GoogleTest binary, a Catch2 binary, a Python script or `cmake -E compare_files` all work. ## enable_testing and where it goes `enable_testing()` makes CMake write a `CTestTestfile.cmake` into each build directory, listing that directory's tests and its subdirectories. `ctest` starts from the top-level build directory, so the call belongs in the **top-level** `CMakeLists.txt`. If only `tests/CMakeLists.txt` calls it, the top-level build directory gets no test file, and a CMake 4.4.4 run of `ctest --test-dir build` printed `No tests were found!!!` while `ctest --test-dir build/tests` still worked. The command must also be called at file scope, not inside a `function()` or `block()`. ## add_test ```cmake add_executable(parser_test parser_test.cpp) add_test(NAME parser_test COMMAND parser_test smoke) ``` - **`NAME`** is the test's name, used for selection and reports. - **`COMMAND`**: if the first word is an executable target, CMake replaces it with the path of the built file, which differs per configuration with multi-config generators. When cross-compiling it also prefixes the target's `CROSSCOMPILING_EMULATOR`. - **`WORKING_DIRECTORY`** defaults to `CMAKE_CURRENT_BINARY_DIR` of the directory that called `add_test`. - **`CONFIGURATIONS`** restricts the test to some configurations. Properties such as `LABELS`, `TIMEOUT` or `WILL_FAIL` are attached afterwards with `set_tests_properties()`, in the same directory that created the test. `add_test` registers a test; it does not build anything, so build before running `ctest`. ## Running ctest ```console cmake --build build ctest --test-dir build --output-on-failure ``` - `--test-dir` (CMake 3.20+) points at the build tree without `cd`. - Test output is captured and hidden unless a test fails and `--output-on-failure` is given, or `-V` is used. - `ctest` exits non-zero when a test fails, which is what CI checks. - With **Visual Studio, Xcode or Ninja Multi-Config**, each test has one command per configuration, so `ctest` needs `-C <config>`. Without it, CMake 4.4.4 reported `Test not available without configuration. (Missing "-C <config>"?)` and marked the test `Not Run`. ## enable_testing() or include(CTest) | | `enable_testing()` | `include(CTest)` | |---|---|---| | Generates test files | yes | yes, by calling `enable_testing()` | | `BUILD_TESTING` option | no | created, `ON` by default; `OFF` skips `enable_testing()` | | Dashboard support (`DartConfiguration.tcl`, CDash submission) | no | yes | | Default per-test timeout | none configured | `DART_TESTING_TIMEOUT`, 1500 seconds | The `CTest` module is meant for projects that submit to a CDash dashboard or want the conventional `BUILD_TESTING` switch: ```cmake include(CTest) if(BUILD_TESTING) add_subdirectory(tests) endif() ``` With `include(CTest)` the timeout matters: a CMake 4.4.4 run with `DART_TESTING_TIMEOUT` set to 2 killed a 4-second test as `Timeout` even under a plain `ctest` run, while the same test passed with `enable_testing()` alone. ## Reading the result A run prints one line per test and a summary. A few outcomes are worth recognising: - `Passed` and `***Failed` follow the exit code. - `***Timeout` means CTest killed the test after its time limit. - `Not Run` means the command could not be started, for example a missing `-C` in a multi-config tree or a test whose fixture setup failed. - `Testing/Temporary/LastTest.log` in the build tree holds every test's full output, even when the console showed only the summary. Tests can be listed without running them with `ctest -N`, which is the quickest way to confirm that `enable_testing()` and `add_test()` did what you expected. ## When the project is a dependency A library pulled in with `add_subdirectory` or `FetchContent` should not add its tests to the parent's suite by default. The `enable_testing` documentation shows the pattern: an `option(Example_ENABLE_TESTING ... ${PROJECT_IS_TOP_LEVEL})` guarding both `enable_testing()` and the `add_test()` calls. How GoogleTest's and Catch2's helpers register one CTest test per test case is covered in the material on those frameworks.
- Why does ctest print Not Run for every test in a Visual Studio build tree?Multi-config generators record one command per configuration, because the executable's path differs between `Debug/` and `Release/`. Without `-C <config>` ctest cannot choose, prints `Test not available without configuration.` and marks each test Not Run. Pass `ctest -C Release`, or set the configuration in a test preset.
- What does a test's exit code have to do with its output, and how can output decide the result instead?By default only the exit code decides: 0 passes, non-zero fails, and captured output is informational. The `PASS_REGULAR_EXPRESSION`, `FAIL_REGULAR_EXPRESSION` and `SKIP_REGULAR_EXPRESSION` test properties let output decide, and `WILL_FAIL` inverts the exit-code result, though a crash such as a segmentation fault still fails.
- Calling enable_testing() only in tests/CMakeLists.txt is enough for ctest at the top
- add_test builds the test executable before running it
- ctest only runs GoogleTest or Catch2 binaries
- include(CTest) and enable_testing() behave identically
- A test passes whenever it prints no error message