skip to content

Property-Based Testing

3 roadmaps5 questions

Property-based testing generates many inputs from an Arb and shrinks any failing case down to a minimal counterexample. Explaining shrinking is what shows you understand why this beats a handful of hand-written cases.

part ofKotlinoverview, primer and where to startread it →
on this pageshow

explore

questions

5

What is property-based testing in Kotest, and how does forAll differ from a traditional example-based test?

level: juniorimportance: must knowfreq 55%

answer

  1. Property = invariant true for ALL inputs
  2. Arb = random + edge cases; Exhaustive = fixed set
  3. forAll returns Boolean; checkAll uses matchers
  4. Default 1000 iterations
  5. Failures shrink to minimal counterexample

basics

~10 s

Instead of hand-picking a few inputs, Kotest generates many random inputs and checks that a rule (a property) holds for all of them. forAll runs your check against hundreds of generated values automatically.

solid answer

~40 s

Property-based testing asserts that a general rule holds across many automatically generated inputs, rather than checking a handful of hardcoded examples. In Kotest you describe inputs with generators (Arb for random, Exhaustive for fixed sets) and run them with forAll or checkAll. forAll takes a lambda returning a Boolean property and passes if every iteration returns true; checkAll takes a lambda using normal matchers (shouldBe) and lets the assertion itself fail. By default Kotest runs ~1000 iterations, mixes in edge cases (0, empty, MIN/MAX), and shrinks any failure to a minimal counterexample. This surfaces bugs that fixed examples miss, like overflow or empty-collection handling, while the shrinker reports the smallest input that breaks the property.

code

kotlin · 14 lines
kotlin
import io.kotest.core.spec.style.StringSpec
import io.kotest.property.Arb
import io.kotest.property.arbitrary.list
import io.kotest.property.arbitrary.int
import io.kotest.property.checkAll
import io.kotest.matchers.shouldBe

class ReverseSpec : StringSpec({
    "reversing twice is identity" {
        checkAll(Arb.list(Arb.int())) { xs ->
            xs.reversed().reversed() shouldBe xs
        }
    }
})

go deeper

for a junior

Defines a property as a rule over many inputs and knows Kotest generates them automatically.

for a middle

Correctly distinguishes forAll (Boolean) from checkAll (matchers) and knows defaults like ~1000 iterations and edge-case injection.

for a senior

Chooses PBT for round-trips/algebraic laws, configures iterations, and explains why richer matcher messages favor checkAll.

for a principal

Frames PBT as part of a testing strategy, weighing flakiness, reproducibility (seeds), and coverage trade-offs against example tests.

## What property-based testing is **Example-based testing** picks specific inputs and asserts specific outputs: `add(2, 3) shouldBe 5`. **Property-based testing (PBT)** instead states an invariant that must hold for *all* valid inputs, then lets the framework generate many inputs to try to falsify it. A **property** is a general truth such as: reversing a list twice yields the original list, or `a + b == b + a` (commutativity). ## Generators: Arb and Exhaustive - **`Arb<T>`** ("arbitrary") produces *random* values plus injected **edge cases**. Examples: `Arb.int()`, `Arb.string()`, `Arb.positiveInt()`. - **`Exhaustive<T>`** enumerates a *fixed, finite* set, e.g. `Exhaustive.boolean()` or `Exhaustive.collection(listOf(...))`. Use it when the input space is small. ## forAll vs checkAll ```kotlin import io.kotest.property.forAll import io.kotest.property.checkAll import io.kotest.matchers.shouldBe // forAll: the lambda RETURNS a Boolean property forAll(Arb.int(), Arb.int()) { a, b -> a + b == b + a // must evaluate to true every iteration } // checkAll: the lambda uses matchers / assertions, returns Unit checkAll(Arb.int(), Arb.int()) { a, b -> (a + b) shouldBe (b + a) } ``` - **`forAll`** expects each iteration to return `true`. If any returns `false`, the test fails. - **`checkAll`** expects you to assert inside (any matcher); the iteration "passes" unless an assertion throws. Both are `suspend` functions, so they integrate with coroutines and can be called from Kotest spec blocks directly. ## How many iterations / what runs By default Kotest runs **1000 iterations** (configurable via `PropTestConfig(iterations = …)`). It deterministically injects **edge cases** first (e.g. `0`, `Int.MIN_VALUE`, `Int.MAX_VALUE`, empty string/list) before random sampling, because those are where bugs hide. ## Shrinking When a property fails, Kotest does not just report the random value that broke it; it **shrinks** the input toward the simplest value that still fails, so you get a minimal, readable counterexample (e.g. it reports `1` rather than `849213`). ## When to reach for PBT Great for round-trips (encode/decode), algebraic laws (associativity, identity), and invariants (output always sorted). Less useful when there is no general rule, only specific known cases.

  • Why prefer checkAll over forAll in many codebases?
    checkAll lets you use the full matcher DSL (shouldBe, shouldContain, soft assertions), giving richer failure messages, whereas forAll only knows pass/fail from a Boolean.
  • How do you change the number of iterations?
    Pass an iteration count or a PropTestConfig to checkAll/forAll, e.g. checkAll(iterations = 5000, Arb.int()) { ... }.

Example tests are spot-checks of a few exam answers; property tests are a rule the whole answer sheet must obey.

saying these in an interview costs you the question

  • Thinks PBT just means looping over a few hardcoded values
  • Believes forAll and checkAll are identical with no difference in return contract
  • Cannot name a single property/invariant example
  • Assumes inputs are purely random with no edge-case injection

context

resolving links…

open as a page

Explain the difference between Arb and Exhaustive generators in Kotest. When would you choose each, and how do you build custom or composed generators?

level: middleimportance: must knowfreq 50%

basics

~10 s

Arb makes random values (plus some edge cases) and suits large input spaces. Exhaustive lists every value and suits small, finite spaces. You build custom ones by mapping, filtering, or combining existing generators.

open as a page

How do you configure iteration count, edge-case injection, seeds, and discards in Kotest property tests, and why do edge cases matter?

level: middleimportance: should knowfreq 35%

basics

~20 s

You can set how many random inputs to try, control the share of special edge values, fix a seed so failures repeat, and limit how many inputs may be skipped. Edge cases catch boundary bugs like zero or empty.

open as a page

What is shrinking in Kotest property tests, why does it matter, and how does it interact with custom Arb generators?

level: seniorimportance: should knowfreq 40%

basics

~20 s

When a property fails, Kotest tries simpler versions of the failing input until it finds the smallest one that still breaks the test. That minimal counterexample makes debugging far easier than a huge random value.

open as a page

When is property-based testing the right tool versus example tests, and how do you design properties (laws, round-trips, oracles) that avoid tautologies and flakiness?

level: principalimportance: nice to knowfreq 22%

basics

~20 s

Use property tests when there's a general rule the code must always obey, like 'encode then decode returns the original'. Pick rules that don't just re-run the code under test, and keep generators deterministic so tests don't flake.

open as a page