skip to content

As a tech lead, how do you decide between withData/table-driven tests and property-based forAll, and what are the trade-offs of each for a given test surface?

level: principalimportance: should knowfreq 24%

answer

  1. Known finite examples -> table/withData
  2. Universal invariant -> property forAll/checkAll
  3. Properties shrink to minimal counterexample
  4. Tables = deterministic, self-documenting regressions
  5. Combine both; control seeds and explosion

basics

~20 s

Use table-driven tests (withData/table) when you have specific, known examples and exact expected outputs. Use property-based forAll when you want many random inputs to check a general rule. Tables document edge cases; properties find unknown ones.

solid answer

~40 s

Table-driven tests (io.kotest.datatest.withData, io.kotest.data.table/forAll) enumerate explicit input->expected rows: precise, deterministic, self-documenting, and ideal for known edge cases, regression cases, and spec examples. Property-based tests (io.kotest.property.forAll with Arb/Gen generators) assert an invariant over many randomly-generated inputs, shrinking failures to a minimal counterexample; they explore the space you didn't think of but require expressing a property rather than a concrete expected value, and can be flakier/slower if generators are unbounded. As a lead I combine them: tables pin down concrete contract examples and past bugs (deterministic CI), while properties guard general laws (round-trip encode/decode, idempotence, commutativity). I watch combinatorial explosion in nested withData, seed control and shrinking in property tests, and keep both readable. Choice hinges on whether the truth is a finite known table or a universal rule.

code

kotlin · 6 lines
kotlin
// round-trip property complements an explicit table of known encodings
test("base64 round-trip") {
    io.kotest.property.checkAll<String> { s ->
        decode(encode(s)) shouldBe s
    }
}

go deeper

for a junior

Knows tables use fixed inputs and properties use random ones.

for a middle

Picks table vs property per case and can write a basic property invariant.

for a senior

Articulates trade-offs (determinism, edge discovery, shrinking, speed) and combines both deliberately.

for a principal

Sets suite-wide strategy: tables for contracts/regressions, properties for laws, with seed control, generator bounds, and explosion guardrails.

## Two complementary tools - **Table-driven** (`withData`, `table(...).forAll`): you **enumerate explicit cases** with concrete expected outputs. - **Property-based** (`io.kotest.property.forAll`/`checkAll` with `Arb`/`Gen`): you assert an **invariant** that must hold for **randomly generated** inputs; Kotest **shrinks** any failure to a minimal counterexample. ## When table-driven wins - The expected output is a **known, finite set** (lookup tables, spec examples, status-code mappings). - You are **pinning regressions** — every past bug becomes a row. - You want **deterministic, self-documenting** CI: the report lists each named case. - The function is hard to express as a general law but easy to enumerate. ## When property-based wins - There is a **universal rule** independent of specific values: `decode(encode(x)) == x` (round-trip), `sort(sort(x)) == sort(x)` (idempotence), `a + b == b + a` (commutativity), ordering/monotonicity, bounds. - You want to **discover unknown edge cases** the team didn't enumerate. - You can express the **oracle** as a property, not a hand-written expected value. ## Trade-offs to manage | Concern | Table-driven | Property-based | |---|---|---| | Determinism | High (fixed rows) | Needs seed control; reproduce via printed seed | | Documentation | Excellent (named cases) | Property statement, less concrete | | Edge discovery | Only what you list | Strong (random + shrinking) | | Speed | Fast, bounded | Can be slow if generators are large | | Failure clarity | Exact row | Shrunk minimal counterexample | ## Practical leadership stance ```kotlin // Table: concrete contract + regressions context("http status mapping") { withData( nameFn = { "${it.first} -> ${it.second}" }, 200 to "OK", 404 to "Not Found", 500 to "Server Error", ) { (code, text) -> statusText(code) shouldBe text } } // Property: universal law test("reverse is involutive") { io.kotest.property.checkAll<List<Int>> { xs -> xs.reversed().reversed() shouldBe xs } } ``` - **Combine**: tables for the concrete contract and bug history; properties for the general laws. - **Guard explosion**: avoid huge nested `withData` cartesian products; bound `Arb` generators and iteration counts. - **Reproducibility**: capture the property **seed** so CI failures are replayable; tables are inherently reproducible. - **Readability**: name table rows; keep properties' invariants crisp and single-purpose. ## Key takeaway Ask: "Is the expected truth a **finite known table** or a **universal rule**?" Enumerate the former with `withData`/`table`, and assert the latter with property `forAll`/`checkAll` — usually you want both.

  • How do you make a failing property test reproducible in CI?
    Kotest prints the random seed on failure; pin/replay that seed (or set a fixed seed) so the exact counterexample regenerates. Tables are already deterministic.
  • Give an invariant better suited to property testing than a table.
    Round-trip laws like decode(encode(x)) == x, or idempotence sort(sort(x)) == sort(x): they hold universally, so enumerating rows is weaker than random generation plus shrinking.

Tables are a checklist of specific cases; properties are a law of physics you test by throwing random objects at it.

saying these in an interview costs you the question

  • Claiming property tests replace all example/table tests
  • Ignoring non-determinism/seed reproducibility in property tests
  • Using random properties where an exact expected lookup is the real spec
  • Building giant nested withData grids instead of a property
  • Not mentioning shrinking as property testing's key advantage

context

resolving links…