In a CMake cross build for an ARM board, how do CMAKE_SYSROOT and the CMAKE_FIND_ROOT_PATH_MODE_* variables keep find_library and find_program from picking the wrong machine's files?
answer
- re-rooted search prefixes
- host root searched by default
- ONLY, NEVER or BOTH
- tools run on host, libraries on target
- prefix paths get re-rooted too
basics
~20 sfind_* commands search the re-rooted prefixes (CMAKE_FIND_ROOT_PATH, CMAKE_SYSROOT) and, by default, the host too. Setting the library, include and package modes to ONLY keeps them inside the target roots, and the program mode to NEVER finds build tools on the host.
solid answer
~40 sIn a cross build, `find_library`, `find_path`, `find_file`, `find_package` and `find_program` **re-root** their search prefixes under `CMAKE_FIND_ROOT_PATH` and `CMAKE_SYSROOT`, then by default also search the plain host paths. That default can let an x86 `libz` from the workstation satisfy a lookup meant for the ARM board, and the failure surfaces only at link time or on the device. The toolchain file therefore sets `CMAKE_FIND_ROOT_PATH_MODE_LIBRARY`, `_INCLUDE` and `_PACKAGE` to `ONLY` (target roots only) and `CMAKE_FIND_ROOT_PATH_MODE_PROGRAM` to `NEVER`, because a code generator or other tool the build runs must be a host executable. Under `ONLY`, a host-absolute `CMAKE_PREFIX_PATH` entry such as `/opt/vendor` is searched as `<sysroot>/opt/vendor`, so target libraries belong inside the sysroot or under an extra `CMAKE_FIND_ROOT_PATH` entry. Individual calls can override the mode with `ONLY_CMAKE_FIND_ROOT_PATH`, `NO_CMAKE_FIND_ROOT_PATH` or `CMAKE_FIND_ROOT_PATH_BOTH`.
code
console · 9 lines# arm-linux.cmake sets CMAKE_SYSROOT to ./sysroot and the LIBRARY mode to ONLY
$ ls sysroot/usr/lib sysroot/opt/vendor/lib
sysroot/opt/vendor/lib:
libfoo.so
sysroot/usr/lib:
libbar.so
$ cmake -S . -B build-arm -G Ninja --toolchain $PWD/arm-linux.cmake -DCMAKE_PREFIX_PATH=/opt/vendor
-- FOO=/work/sysroot/opt/vendor/lib/libfoo.so BAR=/work/sysroot/usr/lib/libbar.sogo deeper
Recall that a cross build has host and target files with the same names, and that the toolchain file tells find commands which to use.
Explain re-rooting, the default search order, and what ONLY, NEVER and BOTH mean for each find command.
Diagnose a host library or ARM tool leaking into a cross build from the cached paths and --debug-find output, and fix it in the toolchain file rather than per call.
Decide how the team provides target libraries, a curated sysroot or a package manager's per-target prefix, so find lookups have one correct place to land.
## Two machines, one search In a cross build the **host** (the x86 workstation running CMake) and the **target** (the ARM board) both have files called `libz.so` and `zlib.h`, and they are not interchangeable. CMake's `find_*` commands were written for native builds, where searching `/usr/lib` is right. For cross builds CMake adds **re-rooting**: - **`CMAKE_SYSROOT`**: one directory holding a copy of the target's root filesystem. It is passed to the compiler as `--sysroot` and prefixed to the search paths. - **`CMAKE_FIND_ROOT_PATH`**: a list of further directories that act as alternative roots, for example a directory of prebuilt ARM libraries. - **Re-rooting**: every search prefix, such as `/usr` or a `CMAKE_PREFIX_PATH` entry, is tried under each root first. `/usr/lib` becomes `<sysroot>/usr/lib`. By default the search order is the `CMAKE_FIND_ROOT_PATH` entries, then `CMAKE_SYSROOT`, then the **non-rooted host directories**. Directories under `CMAKE_STAGING_PREFIX` are never re-rooted, because that variable is always a host path. ## The modes One variable per kind of lookup controls whether the roots, the host, or both are searched: | Mode value | Searches | Typical use | |---|---|---| | `ONLY` | only the re-rooted directories (and below `CMAKE_STAGING_PREFIX`) | libraries, headers, packages for the target | | `NEVER` | only the host paths; the roots are ignored | programs the build runs on the host | | `BOTH` | roots first, then the host paths | the behaviour when nothing is set | The four variables are `CMAKE_FIND_ROOT_PATH_MODE_PROGRAM` (for `find_program`), `CMAKE_FIND_ROOT_PATH_MODE_LIBRARY` (for `find_library`), `CMAKE_FIND_ROOT_PATH_MODE_INCLUDE` (for `find_path` and `find_file`) and `CMAKE_FIND_ROOT_PATH_MODE_PACKAGE` (for `find_package`). The CMake toolchains manual recommends `PROGRAM NEVER` with the other three `ONLY`: includes, libraries and packages come from the target, while executables that must run during the build come from the host. ## What goes wrong without them 1. **A host library satisfies a target lookup.** With `BOTH`, a library missing from the sysroot is found in the host's `/usr/lib` instead. The configure step succeeds, and the cross linker later rejects the x86 file, or a package's config file drags in host include directories. 2. **A target program is chosen as a build tool.** With `find_program` allowed into the sysroot, a code generator found there is an ARM binary the workstation cannot execute, and the custom command that runs it fails. 3. **A vendor directory is not found at all.** Under `ONLY`, `-DCMAKE_PREFIX_PATH=/opt/vendor` is re-rooted to `<sysroot>/opt/vendor`. If the vendor's ARM libraries actually sit on the host at `/home/dev/arm-libs`, add that directory to `CMAKE_FIND_ROOT_PATH` instead, and its `lib/` subdirectory is searched. ## Overriding one call Each `find_*` command accepts options that override the mode for that call: - `ONLY_CMAKE_FIND_ROOT_PATH`: search only the re-rooted directories. - `NO_CMAKE_FIND_ROOT_PATH`: ignore the roots, search the host. - `CMAKE_FIND_ROOT_PATH_BOTH`: search in the default order. A project that needs one host-only lookup, such as a code generator found with `find_program`, can use `NO_CMAKE_FIND_ROOT_PATH` on that call; most projects should leave policy to the toolchain file and not hard-code modes. ## A layout that keeps lookups honest A dependable arrangement for the ARM board: - **The sysroot** is a copy of the board's root filesystem, or the SDK's equivalent, with its `usr/include` and `usr/lib`; it is read-only during builds. - **Extra target libraries** that are not part of the board image, such as a vendor SDK, live in their own directory listed in `CMAKE_FIND_ROOT_PATH`, laid out with `include/` and `lib/` so the standard prefixes resolve under it. - **The staging prefix**, `CMAKE_STAGING_PREFIX`, is where `cmake --install` puts this project's output on the host; because it is never re-rooted, a second project can find what the first installed there. - **Host tools**, such as code generators, come from host directories such as those on `PATH`, the only places `find_program` searches under `PROGRAM NEVER`. With that split, every lookup has one correct place to land, and a wrong path in the cache stands out at once. ## Checking a cross configure - Print the found paths (`message(STATUS "ZLIB: ${ZLIB_LIBRARY}")`) or inspect them in `CMakeCache.txt`: a target library path should lie under the sysroot or a find root. - Run `cmake --debug-find` to log every directory a `find_*` call considered. - Remember that `find_*` results are cached: after fixing the modes, reconfigure a fresh build tree so stale host paths are not reused. How `find_package` chooses between module and config mode, and how imported targets are linked, belong to the material on package discovery; the modes here only decide which filesystem those searches may look at.
- Why is CMAKE_FIND_ROOT_PATH_MODE_PROGRAM normally NEVER rather than ONLY?Programs found with `find_program` are run during the build, by custom commands or by CMake itself, so they must execute on the host. A program inside the sysroot is built for the target and generally cannot run on the workstation. `NEVER` restricts the search to host paths; a project that genuinely needs a target executable path can override one call with `ONLY_CMAKE_FIND_ROOT_PATH`.
- How do you see why find_library chose a particular file in a cross build?Configure with `cmake --debug-find` (or `--debug-find-var=<VAR>` for one result) to log every directory searched, including the re-rooted ones. Because results are cached, delete the cache entry or use a fresh build tree before re-checking, otherwise CMake reuses the earlier answer without searching.
- With a sysroot set, find_library never looks at the host's /usr/lib
- CMAKE_FIND_ROOT_PATH_MODE_PROGRAM should be ONLY so tools come from the target
- CMAKE_PREFIX_PATH entries are searched exactly as written even in ONLY mode
- CMAKE_SYSROOT only affects find_* commands, not the compiler command line
- A wrong host library shows up as a configure error, so a clean configure proves the lookups