skip to content

What is property-based testing in Kotest, and how does forAll differ from a traditional example-based test?

level: juniorimportance: must knowfreq 55%

answer

  1. Property = invariant true for ALL inputs
  2. Arb = random + edge cases; Exhaustive = fixed set
  3. forAll returns Boolean; checkAll uses matchers
  4. Default 1000 iterations
  5. Failures shrink to minimal counterexample

basics

~10 s

Instead of hand-picking a few inputs, Kotest generates many random inputs and checks that a rule (a property) holds for all of them. forAll runs your check against hundreds of generated values automatically.

solid answer

~40 s

Property-based testing asserts that a general rule holds across many automatically generated inputs, rather than checking a handful of hardcoded examples. In Kotest you describe inputs with generators (Arb for random, Exhaustive for fixed sets) and run them with forAll or checkAll. forAll takes a lambda returning a Boolean property and passes if every iteration returns true; checkAll takes a lambda using normal matchers (shouldBe) and lets the assertion itself fail. By default Kotest runs ~1000 iterations, mixes in edge cases (0, empty, MIN/MAX), and shrinks any failure to a minimal counterexample. This surfaces bugs that fixed examples miss, like overflow or empty-collection handling, while the shrinker reports the smallest input that breaks the property.

code

kotlin · 14 lines
kotlin
import io.kotest.core.spec.style.StringSpec
import io.kotest.property.Arb
import io.kotest.property.arbitrary.list
import io.kotest.property.arbitrary.int
import io.kotest.property.checkAll
import io.kotest.matchers.shouldBe

class ReverseSpec : StringSpec({
    "reversing twice is identity" {
        checkAll(Arb.list(Arb.int())) { xs ->
            xs.reversed().reversed() shouldBe xs
        }
    }
})

go deeper

for a junior

Defines a property as a rule over many inputs and knows Kotest generates them automatically.

for a middle

Correctly distinguishes forAll (Boolean) from checkAll (matchers) and knows defaults like ~1000 iterations and edge-case injection.

for a senior

Chooses PBT for round-trips/algebraic laws, configures iterations, and explains why richer matcher messages favor checkAll.

for a principal

Frames PBT as part of a testing strategy, weighing flakiness, reproducibility (seeds), and coverage trade-offs against example tests.

## What property-based testing is **Example-based testing** picks specific inputs and asserts specific outputs: `add(2, 3) shouldBe 5`. **Property-based testing (PBT)** instead states an invariant that must hold for *all* valid inputs, then lets the framework generate many inputs to try to falsify it. A **property** is a general truth such as: reversing a list twice yields the original list, or `a + b == b + a` (commutativity). ## Generators: Arb and Exhaustive - **`Arb<T>`** ("arbitrary") produces *random* values plus injected **edge cases**. Examples: `Arb.int()`, `Arb.string()`, `Arb.positiveInt()`. - **`Exhaustive<T>`** enumerates a *fixed, finite* set, e.g. `Exhaustive.boolean()` or `Exhaustive.collection(listOf(...))`. Use it when the input space is small. ## forAll vs checkAll ```kotlin import io.kotest.property.forAll import io.kotest.property.checkAll import io.kotest.matchers.shouldBe // forAll: the lambda RETURNS a Boolean property forAll(Arb.int(), Arb.int()) { a, b -> a + b == b + a // must evaluate to true every iteration } // checkAll: the lambda uses matchers / assertions, returns Unit checkAll(Arb.int(), Arb.int()) { a, b -> (a + b) shouldBe (b + a) } ``` - **`forAll`** expects each iteration to return `true`. If any returns `false`, the test fails. - **`checkAll`** expects you to assert inside (any matcher); the iteration "passes" unless an assertion throws. Both are `suspend` functions, so they integrate with coroutines and can be called from Kotest spec blocks directly. ## How many iterations / what runs By default Kotest runs **1000 iterations** (configurable via `PropTestConfig(iterations = …)`). It deterministically injects **edge cases** first (e.g. `0`, `Int.MIN_VALUE`, `Int.MAX_VALUE`, empty string/list) before random sampling, because those are where bugs hide. ## Shrinking When a property fails, Kotest does not just report the random value that broke it; it **shrinks** the input toward the simplest value that still fails, so you get a minimal, readable counterexample (e.g. it reports `1` rather than `849213`). ## When to reach for PBT Great for round-trips (encode/decode), algebraic laws (associativity, identity), and invariants (output always sorted). Less useful when there is no general rule, only specific known cases.

  • Why prefer checkAll over forAll in many codebases?
    checkAll lets you use the full matcher DSL (shouldBe, shouldContain, soft assertions), giving richer failure messages, whereas forAll only knows pass/fail from a Boolean.
  • How do you change the number of iterations?
    Pass an iteration count or a PropTestConfig to checkAll/forAll, e.g. checkAll(iterations = 5000, Arb.int()) { ... }.

Example tests are spot-checks of a few exam answers; property tests are a rule the whole answer sheet must obey.

saying these in an interview costs you the question

  • Thinks PBT just means looping over a few hardcoded values
  • Believes forAll and checkAll are identical with no difference in return contract
  • Cannot name a single property/invariant example
  • Assumes inputs are purely random with no edge-case injection

context

resolving links…