When a Kotest property test fails, how does the framework actually produce and search for a smaller counterexample, and what controls how far that search goes?
answer
- sample = value + lazy tree of shrink candidates
- Shrinker<A>.shrink(value) → smaller candidates
- greedy: re-run, descend into first still-failing candidate
- ShrinkingMode: Bounded / Unbounded / Off
- shrink quality lives in the generator
basics
~20 sEach generated sample carries candidate smaller values from its generator's Shrinker. On failure Kotest re-runs the property against those candidates, descends into the first one that still fails, and repeats until no candidate fails. Kotest's ShrinkingMode (Bounded, Unbounded, Off) caps the search.
solid answer
~50 sA kotest-property `Arb` does not just yield a value; it yields a sample plus a lazily-built tree of *smaller* candidates derived from that generator's `Shrinker` (an int shrinks toward zero, a list toward empty, a string toward shorter). When an iteration fails, Kotest walks that tree: it re-runs the property body against candidate values, and whenever a candidate still fails it recurses into that candidate's own candidates. When nothing smaller fails, the last failing value is reported as the minimal counterexample. It is a greedy descent, not an exhaustive search, so "minimal" means "locally minimal for this shrinker". The search is capped by `ShrinkingMode` in `PropTestConfig`: `Bounded(n)` limits the number of steps, `Unbounded` lets it run to completion, `Off` disables it and reports the raw failing value. Shrink quality is a property of the generators: a mapped generator shrinks through its source, while a value assembled by hand in `arbitrary { }` shrinks only as well as the `Shrinker` you supplied.
code
kotlin · 7 lines// Let the search run to completion for the smallest possible counterexample
checkAll(
PropTestConfig(shrinkingMode = ShrinkingMode.Unbounded),
Arb.list(Arb.int(), 0..200),
) { xs ->
xs.sorted().first() shouldBe xs.min()
}go deeper
Know that after a failure Kotest tries smaller inputs and reports the smallest one that still fails.
Describe the Shrinker and the re-run-and-descend loop, and name the ShrinkingMode options that bound it.
Add the operational consequences: the body re-executes many times, results are locally minimal, and shrink quality is determined by how the generator was composed.
Set expectations for the suite: pure property bodies so re-execution is safe, shrinkers invested in the few aggregates that dominate failure output, and an explicit policy on shrink budget for slow tests.
## What shrinking is searching for A property test that fails on `x = 1_849_223_771` tells you almost nothing. The same property failing on `x = 0` tells you where to look. Shrinking is the automatic search from the first value to the smallest value that still reproduces the failure. ## The mechanism **Samples carry candidates.** In kotest-property, drawing from an `Arb` produces a sample that consists of a value plus a lazily-computed tree of *shrink candidates*. Those candidates come from a `Shrinker<A>` — an object whose job is, given a value, to return a list of smaller values worth trying. Built-in shrinkers encode the obvious notion of "smaller" per type: integers move toward zero (halving, then stepping), strings and lists get shorter and their elements simpler, and so on. The tree is lazy: candidates are only computed if a failure actually happens, so passing runs pay nothing. **The search is a greedy descent.** On failure Kotest takes the failing value's candidate list and re-runs the property body for each candidate. The first candidate that also fails becomes the new current value, and the process repeats on *its* candidates. When no candidate fails, the current value is the reported counterexample. Two consequences follow: - **The property body runs many extra times.** Every candidate is a real execution of your test lambda. That matters for slow or side-effecting properties. - **The result is locally minimal, not globally minimal.** The descent follows one path; a different path might have reached something smaller. For most shrinkers this is fine and much cheaper than an exhaustive search. **A step limit exists.** Shrinking a large structure can generate a lot of candidates, and a pathological property can make the descent long. `ShrinkingMode` bounds it: ```kotlin checkAll(PropTestConfig(shrinkingMode = ShrinkingMode.Bounded(500)), Arb.list(Arb.int())) { xs -> ... } ``` - `ShrinkingMode.Bounded(n)` — stop after n steps and report the best value found so far. This is the default shape of the search. - `ShrinkingMode.Unbounded` — keep going until the descent is exhausted. Use when you want the smallest possible counterexample and can pay for it. - `ShrinkingMode.Off` — do not shrink at all; report the original failing input. ## Why some generators shrink badly Shrink quality is decided by the generators, not by the engine: - **`map`** keeps shrinking useful: the source value shrinks and your mapping function is re-applied, so a `UserId` built with `Arb.long().map(::UserId)` shrinks toward `UserId(0)`. - **`filter`** narrows the candidates to those still satisfying the predicate, which is correct but can leave the search little room to move. - **`flatMap`** correlates two values, so shrinking the first changes the generator for the second — the descent is less well-behaved. - **A value assembled by hand in `arbitrary { }`** has no structure the engine can exploit. If you want good counterexamples for such a type, pass a `Shrinker` to the builder; otherwise expect the raw failing value. - **Generators built from a fixed set of values** (picking from a list of constants) have nothing meaningful to shrink toward — that is normal, not a bug. ## Operating it in practice - If counterexamples come back huge and unreadable, the fix is usually in the generator (compose with `map`/`bind`, or supply a `Shrinker`) rather than in the config. - If shrinking is slow, remember that each step re-runs your property body; either make the body cheap or bound the search harder. - If the property has side effects, shrinking re-executes them — see that as a reason to keep property bodies pure, or to switch shrinking off for that test. - Turning shrinking off is legitimate when a shrunk value would be misleading (heavily constrained domain), when the body is expensive, or when you only need to know *that* it fails while you triage. ## What good answers include Name the pieces — `Shrinker`, candidate tree, greedy re-run-and-descend, `ShrinkingMode.Bounded`/`Unbounded`/`Off` — and add the two insights that separate readers from users: shrinking re-executes the property body, and shrink quality lives in the generator, not in the framework.
- Why is the reported counterexample described as minimal only in a local sense?The search is a greedy descent: at each step Kotest takes the first candidate that still fails and recurses into it, rather than exploring every branch of the candidate tree. A different descent path could have reached a smaller value. It stops when no candidate of the current value fails, which makes the result a local minimum for that shrinker, and it may also stop early when a bounded step budget runs out.
- Your property fails and the reported input is exactly as large as the original random sample. What are the likely causes?Either shrinking is switched off for the run, or the generator supplies no useful candidates — a hand-assembled value from arbitrary { } with no Shrinker, or a generator that picks from a fixed set of constants. It can also happen when a tight filter leaves no smaller value satisfying the predicate. The fix is in the generator: compose with map/bind so the components' shrinkers apply, or supply a Shrinker.
saying these in an interview costs you the question
- Thinking shrinking searches for a smaller input without re-running the test body
- Believing the reported counterexample is guaranteed globally minimal
- Assuming every Arb shrinks well regardless of how it was built
- Expecting shrinking to fix a property that is simply wrong about its own precondition
- Treating a long shrink phase as a framework bug rather than a bounded search over an expensive body