How does Kotest's assertSoftly actually aggregate failures — what does it collect, what does it report, and which failures inside the block does it NOT aggregate?

seniormust knowfreq 46%

answer

error collector mode: hard → soft
matchers report to the collector, not throw
one MultiAssertionError at block end
NPE / non-Kotest assertion aborts, not collected
nested blocks: only the outermost throws

basics

~20 s

Inside the block Kotest switches its error collector to soft mode, so matcher failures are recorded instead of thrown; at the end it throws one MultiAssertionError listing them all. Only failures routed through Kotest's collector are aggregated — non-assertion exceptions abort the block immediately.

solid answer

~50 s

assertSoftly { } flips Kotest's error collector into soft mode for the duration of the block. Kotest matchers do not throw directly — they hand the failure to that collector, which in soft mode records it and lets execution continue. When the block ends, Kotest restores hard mode and, if anything was collected, throws a single MultiAssertionError whose message lists every failure, numbered, so one run tells you all three fields that are wrong instead of only the first.

What it does not aggregate: anything that does not go through the collector. A NullPointerException, an IndexOutOfBoundsException from your code under test, or a non-Kotest assertion library's AssertionError propagates out and aborts the rest of the block, so later assertions never run. Nested assertSoftly calls do not each throw — only the outermost block reports. On the JVM the collector is per-thread, so assertions executed on another thread inside the block are not collected softly.

`assertSoftly { }` flips Kotest's error collector into soft mode for the duration of the block. Kotest matchers do not throw directly — they hand the failure to that collector, which in soft mode **records** it and lets execution continue. When the block ends, Kotest restores hard mode and, if anything was collected, throws a single `MultiAssertionError` whose message lists every failure, numbered, so one run tells you all three fields that are wrong instead of only the first. What it does **not** aggregate: anything that does not go through the collector. A `NullPointerException`, an `IndexOutOfBoundsException` from your code under test, or a non-Kotest assertion library's `AssertionError` propagates out and **aborts the rest of the block**, so later assertions never run. Nested `assertSoftly` calls do not each throw — only the outermost block reports. On the JVM the collector is per-thread, so assertions executed on another thread inside the block are not collected softly.

code

kotlin · 6 lines

kotlin

assertSoftly(response) {
    status shouldBe 200
    contentType shouldBe "application/json"
    body.userId shouldBe 42
}
// one MultiAssertionError listing every failed line, not just the first

go deeper

for a junior

Know that it reports every failed matcher in the block at once instead of stopping at the first.

for a middle

Explain the collector mode switch, the single MultiAssertionError thrown at block end, and the receiver-taking overload.

for a senior

Add the non-aggregated cases — plain exceptions, non-Kotest assertions, other threads — plus nesting semantics and the independent-vs-dependent judgement.

for a principal

Set a suite-wide convention: fail-fast guards for preconditions, soft blocks for independent field checks, clues inside loops, and no side effects in assertion blocks.

The mechanism

Kotest matchers do not call throw themselves. Every matcher funnels its result through a per-test error collector, which has two modes:

hard (the default): a failure is thrown immediately — the familiar fail-fast behaviour;
soft: a failure is appended to a list and execution continues.

assertSoftly { ... } sets the mode to soft, runs the block, then restores the previous mode and, if the list is non-empty, throws one aggregate error — io.kotest.assertions.MultiAssertionError — whose message enumerates the collected failures ("The following N assertions failed:" followed by each numbered message and the source location).

That indirection is the whole design: soft assertions are not a try/catch around your matchers, they are a mode change in the reporting channel. It explains every behaviour below.

Why it exists

Fail-fast reporting gives you one defect per test run. Asserting six fields of a response object means six red-green cycles if five of them are wrong. assertSoftly turns that into one cycle: you see all six results at once. It is most valuable when the assertions are independent observations of the same state — the fields of a DTO, the headers of a response, the rows of a projection.

Kotest also offers a receiver-taking overload so the object under test becomes the receiver:

kotlin

assertSoftly(user) {
    name shouldBe "Ada"
    email shouldBe "[email protected]"
}

What is NOT aggregated

This is the part that separates users from people who understand the mechanism.

1. Non-assertion exceptions. If the block throws a NullPointerException, a ClassCastException, or any exception raised by the code under test, that exception is not a collected assertion failure — it escapes the block. Everything after it in the block is skipped. The practical consequence is the one people trip over: soft assertions do not make a block exception-proof. If assertion #2 dereferences a null that assertion #1 just proved was null, you still lose assertions #3 onward.

2. Assertions that bypass Kotest's collector. A raw kotlin.assert(...), a hand-written if (x != y) throw AssertionError(...), or another assertion library's failure does not pass through the collector — it throws directly and aborts the block. Mixing assertion libraries inside assertSoftly therefore silently defeats it.

3. Failures on other threads. On the JVM the collector is thread-scoped. Assertions executed on a different thread inside the block do not see soft mode. This matters for tests that assert inside a callback executed by a pool.

Nesting

assertSoftly blocks nest safely. If the mode is already soft when the function is entered, the inner call does not re-arm or throw on its own — collection continues and only the outermost block throws the aggregate. So extracting a group of assertions into a helper that itself uses assertSoftly does not fragment the report into several errors.

Interaction with clues and exception assertions

Clues (withClue / asClue) compose with soft mode: each collected failure is captured with the clue context active at the moment it was recorded, so an aggregate report can carry a different clue per line. That combination — a soft block iterating a collection with withClue(item) inside — is the idiomatic way to get "which element failed and why" for every element in one run.

shouldThrow inside a soft block behaves normally: its own failure is a matcher failure and gets collected, but a non-matching exception escaping the block is still a plain exception and aborts, per rule 1.

Cost and when not to use it

Soft assertions delay feedback for dependent assertions. If assertion #1 establishes a precondition for #2 ("the list is non-empty" then "the first element is X"), aggregating them produces a cascade of derivative failures — one real defect reported as five, which is noise, not information. The judgement is: aggregate independent observations, fail fast on dependent ones. A common shape is a fail-fast guard followed by a soft block for the field-by-field checks.

A second cost: because the block keeps running, code after a failed assertion executes against state you have just proved is wrong. That is usually harmless for pure reads, and occasionally confusing when the assertions have side effects — one more reason to keep assertion blocks side-effect free.

Interview framing

A weak answer says "it collects all failures instead of stopping at the first". A strong one names the collector mode change, MultiAssertionError as the single aggregate thrown at the end, the nesting rule, and — the discriminator — that non-assertion exceptions and non-Kotest assertions are not aggregated and will truncate the block.

## The mechanism Kotest matchers do not call `throw` themselves. Every matcher funnels its result through a per-test **error collector**, which has two modes: - **hard** (the default): a failure is thrown immediately — the familiar fail-fast behaviour; - **soft**: a failure is appended to a list and execution continues. `assertSoftly { ... }` sets the mode to soft, runs the block, then restores the previous mode and, if the list is non-empty, throws one aggregate error — `io.kotest.assertions.MultiAssertionError` — whose message enumerates the collected failures ("The following N assertions failed:" followed by each numbered message and the source location). That indirection is the whole design: soft assertions are not a try/catch around your matchers, they are a *mode change in the reporting channel*. It explains every behaviour below. ## Why it exists Fail-fast reporting gives you one defect per test run. Asserting six fields of a response object means six red-green cycles if five of them are wrong. `assertSoftly` turns that into one cycle: you see all six results at once. It is most valuable when the assertions are **independent observations of the same state** — the fields of a DTO, the headers of a response, the rows of a projection. Kotest also offers a receiver-taking overload so the object under test becomes the receiver: ```kotlin assertSoftly(user) { name shouldBe "Ada" email shouldBe "[email protected]" } ``` ## What is NOT aggregated This is the part that separates users from people who understand the mechanism. **1. Non-assertion exceptions.** If the block throws a `NullPointerException`, a `ClassCastException`, or any exception raised by the code under test, that exception is not a collected assertion failure — it escapes the block. Everything after it in the block is skipped. The practical consequence is the one people trip over: soft assertions do **not** make a block exception-proof. If assertion #2 dereferences a null that assertion #1 just proved was null, you still lose assertions #3 onward. **2. Assertions that bypass Kotest's collector.** A raw `kotlin.assert(...)`, a hand-written `if (x != y) throw AssertionError(...)`, or another assertion library's failure does not pass through the collector — it throws directly and aborts the block. Mixing assertion libraries inside `assertSoftly` therefore silently defeats it. **3. Failures on other threads.** On the JVM the collector is thread-scoped. Assertions executed on a different thread inside the block do not see soft mode. This matters for tests that assert inside a callback executed by a pool. ## Nesting `assertSoftly` blocks nest safely. If the mode is already soft when the function is entered, the inner call does not re-arm or throw on its own — collection continues and only the **outermost** block throws the aggregate. So extracting a group of assertions into a helper that itself uses `assertSoftly` does not fragment the report into several errors. ## Interaction with clues and exception assertions Clues (`withClue` / `asClue`) compose with soft mode: each collected failure is captured **with the clue context active at the moment it was recorded**, so an aggregate report can carry a different clue per line. That combination — a soft block iterating a collection with `withClue(item)` inside — is the idiomatic way to get "which element failed and why" for every element in one run. `shouldThrow` inside a soft block behaves normally: its own failure is a matcher failure and gets collected, but a *non-matching* exception escaping the block is still a plain exception and aborts, per rule 1. ## Cost and when not to use it Soft assertions delay feedback for **dependent** assertions. If assertion #1 establishes a precondition for #2 ("the list is non-empty" then "the first element is X"), aggregating them produces a cascade of derivative failures — one real defect reported as five, which is noise, not information. The judgement is: aggregate independent observations, fail fast on dependent ones. A common shape is a fail-fast guard followed by a soft block for the field-by-field checks. A second cost: because the block keeps running, code after a failed assertion executes against state you have just proved is wrong. That is usually harmless for pure reads, and occasionally confusing when the assertions have side effects — one more reason to keep assertion blocks side-effect free. ## Interview framing A weak answer says "it collects all failures instead of stopping at the first". A strong one names the collector mode change, `MultiAssertionError` as the single aggregate thrown at the end, the nesting rule, and — the discriminator — that non-assertion exceptions and non-Kotest assertions are *not* aggregated and will truncate the block.

You wrap a whole test in assertSoftly and a NullPointerException is thrown halfway through. Do you still get the earlier assertion failures?
The block stops at the NPE, so every assertion after it is skipped — soft mode only governs failures routed through Kotest's error collector, and an NPE from the code under test is not one. Treat `assertSoftly` as a reporting aggregator for independent matcher checks, not as an exception shield; put a fail-fast null/emptiness guard before the soft block.
Does nesting assertSoftly inside another assertSoftly produce two separate errors?
No. If soft mode is already active, the inner call simply continues collecting into the same collector and does not throw on its own; only the outermost block restores hard mode and throws the aggregate. That is what makes it safe to extract groups of assertions into helper functions that themselves use `assertSoftly`.
When would you deliberately NOT use soft assertions?
When the assertions are dependent — a later check is only meaningful if an earlier one held. Aggregating those turns one real defect into a cascade of derived failures that obscures the cause. Fail fast on preconditions and structural invariants, and aggregate only independent observations of the same state, such as the fields of a response.

saying these in an interview costs you the question

Believing assertSoftly catches all exceptions, including NPEs from the code under test
Expecting one error per failed assertion instead of a single aggregate MultiAssertionError
Thinking nested assertSoftly blocks each throw their own error
Assuming non-Kotest assertions (kotlin.assert, another library) participate in soft mode
Wrapping dependent assertions and then debugging a cascade of derived failures

How does Kotest's assertSoftly actually aggregate failures — what does it collect, what does it report, and which failures inside the block does it NOT aggregate?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

The mechanism

Why it exists

What is NOT aggregated

Nesting

Interaction with clues and exception assertions

Cost and when not to use it

Interview framing

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions7 questions

▾related topics3 topics

The mechanism

Why it exists

What is NOT aggregated

Nesting

Interaction with clues and exception assertions

Cost and when not to use it

Interview framing

answer

go deeper

context