skip to content

In Kotest's property-testing module, what is the difference in contract between `checkAll`, `forAll` and `forNone`, and how does each one decide that a property has failed?

level: middleimportance: must knowfreq 45%

answer

  1. checkAll = Unit lambda + matchers
  2. forAll = Boolean predicate
  3. forNone = fails on first true
  4. first failure aborts, then shrinks
  5. matchers in forAll → type error

basics

~20 s

checkAll's lambda returns Unit and fails when an assertion inside it throws. forAll's lambda must return Boolean and fails when it returns false. forNone is the inverse — it fails when the predicate returns true.

solid answer

~50 s

Kotest's property module exposes two different contracts. - **`checkAll`** runs an *assertion block*: the lambda returns `Unit`, and the property fails when a matcher or any assertion inside it throws. The failure report therefore carries the matcher's own message (expected vs actual) plus the offending inputs. - **`forAll`** runs a *predicate*: the lambda must return `Boolean`, and the property fails on the first `false`. You learn *which* inputs broke it, but not *why* — there is no expected/actual detail. - **`forNone`** is `forAll`'s complement: it asserts the predicate is `false` for every generated value and fails on the first `true`. All three iterate the configured number of times, abort on the first failure (unless `maxFailure` is raised via `PropTestConfig`), shrink the failing input, and print the shrunk value plus the seed. In production suites I default to `checkAll` for the diagnostics; `forAll` reads well for pure algebraic laws.

code

kotlin · 3 lines
kotlin
checkAll(Arb.int(), Arb.int()) { a, b ->
    (a + b) shouldBe (b + a)
}

go deeper

for a junior

Recall the mechanical difference: checkAll = assertions inside, forAll = return true/false, forNone = must always be false.

for a middle

Explain why the contracts differ in practice — matcher messages survive into the failure report with checkAll, while forAll only tells you which inputs broke it — and show a small example of each.

for a senior

Argue the default (checkAll for debuggability, forAll for one-line laws), mention that the run aborts on first failure and shrinks, and flag the matcher-in-forAll type trap and the inspectors name collision.

for a principal

Frame it as a diagnostics policy for the suite: which contract the team standardises on, how failure messages are expected to read for on-call debugging, and where boolean laws are still worth the terseness.

## Two contracts, one engine Kotest's property testing (artifact `kotest-property`, package `io.kotest.property`) drives the same machinery in every case: build a stream of values from the supplied generators, run the body once per value, and on failure shrink the input down to a minimal counter-example and report it. What differs between the entry points is **the contract of the lambda you hand it** — and therefore how the engine learns that this iteration failed. ## `checkAll` — the assertion contract `checkAll` takes a lambda returning `Unit`. The body is an ordinary test body: you use Kotest matchers (`shouldBe`, `shouldContain`, `shouldThrow`, …) or any assertion mechanism that throws on failure. An iteration is a failure exactly when the block throws. ```kotlin checkAll(Arb.int(), Arb.int()) { a, b -> (a + b) shouldBe (b + a) } ``` Because the failure is an exception, the exception's message travels into the property failure report. You see the matcher's expected/actual diff *and* the inputs that produced it. That is the whole reason `checkAll` is the default choice in real suites: a red property test tells you what was wrong, not merely that something was. ## `forAll` / `forNone` — the predicate contract `forAll` takes a lambda returning `Boolean`. The property holds if every iteration returns `true`; it fails at the first `false`. ```kotlin forAll(Arb.string(), Arb.string()) { a, b -> (a + b).length == a.length + b.length } ``` `forNone` is the mirror image: it asserts the predicate is `false` for *every* generated value, failing at the first `true`. It expresses "no input in this space satisfies P" — e.g. no value produced by a sanitiser generator still contains a control character. Semantically `forNone { p }` and `forAll { !p }` agree, but `forNone` states the intent and reports it that way. The cost of the predicate contract is diagnostic poverty. When a boolean property fails you get the (shrunk) inputs and "property failed", nothing about which sub-expression was wrong. For a one-line law that is fine; for a multi-step property it is a debugging tax. ## Failure detection, shrinking and reporting On failure the engine does the same thing for all three: it records the failing inputs, runs shrinking to find a smaller input that still fails, and raises an `AssertionError` describing the property (attempt number, shrunk inputs, the underlying cause when there was one, and the seed to reproduce the run). By default the run **stops at the first failure** — it does not keep going and collect every failing input. That behaviour is only changed by raising `maxFailure` on `PropTestConfig`, which lets the run tolerate a budget of failures before it gives up. ## A type-system trap worth knowing Kotest matchers are not `Boolean`-returning — `shouldBe` returns the receiver value. So writing an assertion as the last expression of a `forAll` block is usually a *compile* error (`Unit`/`T` where `Boolean` is expected), which is the compiler telling you that you wanted `checkAll`. The one dangerous case is a property over `Boolean` values, where the types line up and the assertion result silently becomes the predicate. If your lambda contains matchers, use `checkAll`. Also note the name collision: `io.kotest.inspectors.forAll` is the *inspector* that asserts something about every element of an existing collection, and has nothing to do with property testing. If `forAll` behaves unexpectedly, check the import. ## Choosing between them - Use **`checkAll`** by default: for anything with more than one assertion, anything where the expected value is computed, and anything a colleague will have to debug at 3am. - Use **`forAll`** for compact algebraic laws — commutativity, associativity, idempotence, round-trips — where the boolean *is* the law and reads better than an assertion. - Use **`forNone`** to state a negative invariant explicitly, so the intent survives in the test name and the failure message. Both families accept the same configuration surface (explicit iteration counts, a `PropTestConfig`), the same generators, and the same arity overloads, so switching between them is mechanical — the choice is purely about how much the failure message needs to say.

  • What happens if you put a Kotest matcher such as `shouldBe` inside a `forAll` block?
    Usually it will not compile: matchers return the receiver (or `Unit`-like values), while `forAll` requires a `Boolean` result, so the compiler rejects the lambda. That error is the signal to switch to `checkAll`. The exception is a property whose values are already `Boolean`, where the types accidentally line up and the assertion's result becomes the predicate — a silent and confusing bug.
  • Does a failing `checkAll` report every failing input it found, or just one?
    Just one by default. The run aborts on the first failing iteration, shrinks that input to a minimal counter-example, and reports it together with the underlying assertion error and the seed. Only by raising `maxFailure` on `PropTestConfig` does the run continue past failures, and even then the report is about the failure budget being exceeded rather than an exhaustive failure list.
  • You imported `forAll` and it complains about a collection instead of running a property. What went wrong?
    You imported `io.kotest.inspectors.forAll`, the inspector that asserts a block holds for every element of an existing collection, instead of the property-testing `forAll` from `io.kotest.property`. The two share a name and are easy to auto-import wrongly; fixing the import resolves it.

saying these in an interview costs you the question

  • Saying `checkAll` returns a Boolean or that you must return `true` from it
  • Claiming `forAll` accepts matchers/assertions the same way `checkAll` does
  • Believing a property test runs all iterations and reports every failure by default
  • Thinking `forNone` means "run zero iterations" or "skip the property"
  • Confusing the inspector `io.kotest.inspectors.forAll` with the property-testing `forAll`

context