skip to content

In Catch2 3, how do you compare a computed double against an expected value, and when do WithinRel and WithinAbs differ?

level: middleimportance: must knowfreq 40%

answer

  1. never == on a computed double
  2. REQUIRE_THAT plus a matcher
  3. relative scales with magnitude
  4. relative to zero means exactly zero

basics

~20 s

Use REQUIRE_THAT with Catch::Matchers::WithinRel or WithinAbs instead of ==. WithinRel scales its tolerance with the values' magnitude, defaulting to 100 machine epsilons; WithinAbs uses a fixed margin, which is what works when the expected value is zero.

solid answer

~40 s

A computed `double` is rarely bit-identical to a decimal literal: a triangle area of 0.06 on paper came out as `0.05999999999999999`, so `REQUIRE(area == 0.06)` fails. Catch2 3's preferred tool is a floating-point **matcher** from `catch2/matchers/catch_matchers_floating_point.hpp`, used with `REQUIRE_THAT(value, matcher)`. `WithinRel(target, eps)` passes when `|arg - target| <= eps * max(|arg|, |target|)`, so the tolerance scales with the magnitude; with no `eps` it uses `std::numeric_limits<double>::epsilon() * 100`. `WithinAbs(target, margin)` passes when the difference is at most `margin`. They diverge near zero: relative to a target of 0 the allowed difference is `eps * |arg|`, which only an exact zero satisfies, so a residue of `5.5e-17` fails `WithinRel(0.0)` but passes `WithinAbs(0.0, 1e-12)`. Matchers combine with `||` when both regimes matter. `Catch::Approx` still exists, but the documentation recommends against it for new code.

code

cpp · 25 lines
cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_floating_point.hpp>
#include <cmath>
#include <stdexcept>

struct Point { double x, y; };
double triangleArea(Point a, Point b, Point c) {
    double twice = (b.x - a.x) * (c.y - a.y) - (c.x - a.x) * (b.y - a.y);
    if (twice == 0.0) throw std::invalid_argument("degenerate triangle");
    return std::fabs(twice) / 2.0;
}

using Catch::Matchers::WithinAbs;
using Catch::Matchers::WithinRel;

TEST_CASE("triangle area is compared with a tolerance", "[geometry]") {
    double area = triangleArea({0.1, 0.1}, {0.7, 0.1}, {0.1, 0.3});  // 0.06 on paper
    CHECK(area == 0.06);                       // fails: 0.05999999999999999 == 0.06
    CHECK_THAT(area, WithinRel(0.06));         // passes with the default eps
    CHECK_THAT(area, WithinRel(0.06, 1e-9));   // passes with an explicit eps

    double flat = 0.1 + 0.2 - 0.3;             // about 5.5e-17, not zero
    CHECK_THAT(flat, WithinRel(0.0));          // fails: relative to zero
    CHECK_THAT(flat, WithinAbs(0.0, 1e-12));   // passes: absolute margin
}

go deeper

for a junior

Recall that computed doubles are compared with a tolerance, and that Catch2 3 does this with REQUIRE_THAT and WithinRel or WithinAbs.

for a middle

Explain the relative formula, the default epsilon, and why a relative tolerance can never pass a nonzero value against a target of zero.

for a senior

Pick tolerances from the computation, combine relative and absolute matchers where values span zero, and avoid the matcher lifetime trap when storing combinations.

for a principal

Agree a team convention for numeric tolerances and where they are documented, so a loosened bound in review is a visible decision rather than a quiet fix for a red build.

## Why == fails on computed doubles Most decimal fractions have no exact binary representation, and every arithmetic step rounds. A triangle with vertices `(0.1, 0.1)`, `(0.7, 0.1)` and `(0.1, 0.3)` has an area of exactly 0.06 on paper; computed as half the absolute cross product in `double`, it came out as `0.05999999999999999`. Catch2's decomposition makes the failure obvious: ```plaintext e.cpp:10: FAILED: CHECK( area == 0.06 ) with expansion: 0.05999999999999999 == 0.06 ``` The test is right to want 0.06; it is wrong to demand the same bits. Why rounding happens is a numerical-analysis subject; the question here is which Catch2 tool expresses "close enough". ## Floating-point matchers Catch2 3 documents **matchers** as the preferred way to compare floating-point numbers. They live in `catch2/matchers/catch_matchers_floating_point.hpp` and are used with `REQUIRE_THAT(value, matcher)` or `CHECK_THAT(value, matcher)`: | Matcher | Passes when | Default | |---|---|---| | `WithinRel(target, eps)` | `abs(arg - target) <= eps * max(abs(arg), abs(target))` | `eps` = `numeric_limits<T>::epsilon() * 100` | | `WithinAbs(target, margin)` | `abs(arg - target) <= margin` | none; margin is required | | `WithinULP(target, maxUlpDiff)` | at most `maxUlpDiff` representable values apart | none; requires IEEE-754 | | `IsNaN()` | the value is a NaN | added in Catch2 3.3.2 | Details from the 3.16.1 source and documentation: - `WithinRel` is **symmetric**: swapping the argument and the target gives the same answer. - `WithinRel` refuses an `eps` below 0 or at or above 1, and `WithinAbs` refuses a negative margin, by throwing at construction. - `WithinAbs` has only a `double` overload, since a `float` converts to `double` exactly. - Each matcher describes itself in the report, for example `0.05999999999999999 and 0.06 are within 2.22045e-12% of each other`. ## Relative versus absolute The two express different promises: 1. **Relative** tolerance says "agree to about this many significant digits". It is right for values of arbitrary magnitude, such as areas that may be 0.06 or 6e6, because the allowed error grows with the value. 2. **Absolute** tolerance says "within this distance". It is right when the scale is known and fixed, and it is the **only** one of the two that works at zero. The zero case surprises people. With a target of 0, the relative bound becomes `eps * abs(arg)`, and `abs(arg) <= eps * abs(arg)` holds only for `arg == 0` because `eps < 1`. A residue such as `0.1 + 0.2 - 0.3`, about `5.5e-17`, therefore fails `WithinRel(0.0)` but passes `WithinAbs(0.0, 1e-12)`. When a value may be large or near zero, combine them: ```cpp REQUIRE_THAT(area, WithinRel(expected, 1e-9) || WithinAbs(0.0, 1e-12)); ``` ## Approx, and why not `Catch::Approx` from `catch2/catch_approx.hpp` is the older mechanism, used inside an ordinary comparison: `REQUIRE(area == Catch::Approx(0.06))`. The documentation strongly recommends against it in new code and calls it deprecated, although the type carries no `[[deprecated]]` attribute. Its documented problems: - Its internal computation is always in `double`, so `float` inputs behave slightly differently. - Its relative comparison is **not symmetric**: `Approx(10).epsilon(0.1) != 11.1` while `Approx(11.1).epsilon(0.1) == 10`. - By default it uses only a relative margin, so `Approx(0) == x` passes only for `x == 0`. ## float, double and what the report shows - `WithinRel` has `float` and `double` overloads, and the default `eps` follows the argument's type: for a `float` target it is `std::numeric_limits<float>::epsilon() * 100`, which is far looser than the `double` default. - `WithinULP` also works on both types, counting steps in the type of its target. - `WithinAbs` takes `double` only, so a `float` value is widened before comparison. - In the report, the matcher's description replaces the `a == b` expansion. A passing `WithinRel` with the default tolerance printed `0.05999999999999999 and 0.06 are within 2.22045e-12% of each other`, which is the default `eps` expressed as a percentage, not the actual difference between the values. - A combined matcher prints both descriptions joined by `and` or `or`, so a reader sees every bound that was checked. ## Choosing a tolerance - Derive it from the computation, not from trial and error: a few operations on well-scaled inputs justify a tight relative bound; long accumulations or subtraction of nearly equal values need a looser one. - Use `WithinAbs` for quantities whose expected value is zero or whose scale is fixed by the domain, such as a residual or an angle in radians. - Keep the tolerance next to the assertion, so a reviewer sees the promise the test makes.

  • Why is storing a combined Catch2 matcher in a variable dangerous?
    The combining operators `&&`, `||` and `!` do not take ownership of the matchers they combine; they hold references. Writing `auto m = WithinAbs(0, 0.02) && !WithinULP(0., 1);` makes `m` refer to the temporary created by `!`, which is destroyed at the end of the statement, so using `m` later is a use-after-free. Combine matchers inline in the assertion, or store each part in its own named variable first.
  • When would you choose Catch2's WithinULP over WithinRel?
    When the requirement is stated in representable steps, for example a function documented as correctly rounded or within one ULP of the exact result. `WithinULP(target, n)` passes when at most `n` representable values separate the two, which is independent of magnitude without choosing a decimal `eps`. It requires IEEE-754 floating point, and Catch2's ULP distance treats `-0.0` and `0.0` as zero apart.

  • REQUIRE(area == 0.06) is fine because 0.06 is a short decimal.
  • WithinRel(0.0) accepts tiny values close to zero.
  • WithinAbs scales its margin with the size of the values.
  • Catch::Approx is the recommended way to compare doubles in Catch2 3.
  • A relative tolerance of 1e-9 is a safe default for any computation.

Assumes Catch2 3, where floating-point matchers live in catch2/matchers/catch_matchers_floating_point.hpp and the documentation recommends them over Catch::Approx.