skip to content

Matchers & Assertions

Kotest's assertion library is an infix matcher DSL spanning equality, collections, strings, types, and exceptions, plus soft assertions and failure clues. Interviewers dig here to see whether you can produce readable failure output and extend the DSL with your own matchers.

on this pageshow

explore

questions

19

In Kotest, what exactly is the difference between shouldContainExactly, shouldContainExactlyInAnyOrder and shouldContainAll — particularly with respect to ordering, size and duplicate elements?

level: middleimportance: must knowfreq 45%

answer

  1. Exactly = positional
  2. InAnyOrder = multiset, not set
  3. Duplicate counts still matter in InAnyOrder
  4. All = subset, extras allowed
  5. Weak pair: size + contains

basics

~20 s

shouldContainExactly requires the same elements in the same order and the same size. shouldContainExactlyInAnyOrder requires the same multiset — same size and same duplicate counts, order irrelevant. shouldContainAll is only a subset check: extras and duplicate mismatches are ignored.

solid answer

~50 s

They form a strength ladder. - `shouldContainExactly(1, 2, 2)` — positional comparison. Same size, same elements, same order, duplicates counted. `listOf(2, 1)` fails against `listOf(1, 2)`. - `shouldContainExactlyInAnyOrder(1, 2, 2)` — multiset comparison. Order is irrelevant, but size and multiplicity are not: `listOf(1, 1, 2)` does **not** satisfy `shouldContainExactlyInAnyOrder(1, 2, 2)`. This is the matcher people wrongly describe as a set comparison. - `shouldContainAll(1, 2)` — containment only. The actual collection may contain extra elements and any number of duplicates; it just has to include each named element at least once. - `shouldContain(1)` — single-element membership. Pick the strongest one the contract justifies. Use `Exactly` when order is part of the API contract (a sorted query, a pipeline output), `ExactlyInAnyOrder` when the source has no defined order, and `All` only when you deliberately want a partial assertion.

code

kotlin · 7 lines
kotlin
listOf(1, 2, 3) shouldContainExactly listOf(1, 2, 3)            // pass
listOf(1, 2, 3) shouldContainExactly listOf(3, 2, 1)            // fail: order

listOf(1, 2) shouldContainExactlyInAnyOrder listOf(2, 1)         // pass
listOf(1, 1, 2) shouldContainExactlyInAnyOrder listOf(1, 2, 2)   // fail: duplicate counts

listOf(1, 2, 3, 4) shouldContainAll listOf(1, 3)                 // pass: subset only

go deeper

for a junior

Recall the three-way split: exact order, same elements any order, and subset containment.

for a middle

Nail the multiset semantics of InAnyOrder with a duplicate example, and explain that Exactly also pins size and order.

for a senior

Frame it as assertion strength: pick the strongest matcher the contract guarantees, and treat a flaking Exactly as evidence the order is undefined rather than as a reason to weaken the test.

for a principal

Discuss it as a suite-wide convention — undefined ordering should be expressed once in the API contract and reflected consistently in the assertions, not rediscovered per test.

## The ladder Kotest's collection matchers differ along three axes: does order matter, does size matter, do duplicates matter. Placing each matcher on those axes is the whole question. | Matcher | Order | Size/extras | Duplicates | |---|---|---|---| | `shouldContainExactly` | must match | must match | counted, positionally | | `shouldContainExactlyInAnyOrder` | irrelevant | must match | counted | | `shouldContainAll` | irrelevant | extras allowed | ignored | | `shouldContain` | irrelevant | extras allowed | ignored (single element) | ## shouldContainExactly This is a full positional comparison — conceptually the same statement as `shouldBe` on two lists, but expressed as a collection assertion and reported as one. Same length, same elements, same order. ```kotlin listOf(1, 2, 3) shouldContainExactly listOf(1, 2, 3) // passes listOf(1, 2, 3) shouldContainExactly listOf(3, 2, 1) // fails: order listOf(1, 2) shouldContainExactly listOf(1, 2, 2) // fails: size ``` When it fails, Kotest reports the divergence — which elements are missing from the actual collection and which are present but unexpected — rather than just printing two lists and leaving you to diff them. ## shouldContainExactlyInAnyOrder The most misunderstood one. It is a **multiset** (bag) comparison, not a set comparison. Two collections match when they have the same size and every element occurs the same number of times in each. ```kotlin listOf(1, 2) shouldContainExactlyInAnyOrder listOf(2, 1) // passes listOf(1, 1, 2) shouldContainExactlyInAnyOrder listOf(1, 2, 2) // FAILS listOf(1, 1, 2) shouldContainExactlyInAnyOrder listOf(1, 2) // FAILS (size) ``` The second example is the one candidates get wrong: both sides contain exactly the values {1, 2}, so anyone thinking "set comparison" expects a pass. Duplicate counts differ, so it fails. That is a feature — a de-duplication bug in the code under test should not slip past an "any order" assertion. If you truly want set semantics, convert explicitly: `actual.toSet() shouldContainExactlyInAnyOrder expected.toSet()`, and be aware you have just stopped testing duplicates at all. ## shouldContainAll A containment/subset assertion. Every named element must be present at least once; the actual collection may contain anything else besides. ```kotlin listOf(1, 2, 3, 4) shouldContainAll listOf(1, 3) // passes listOf(1, 1, 1) shouldContainAll listOf(1) // passes ``` This is a genuinely weak assertion, and that is sometimes right — asserting that an audit log contains two specific events without pinning the rest. But it is often used as a way to make a flaky test green, and then the test no longer detects extra, duplicated or wrongly ordered output. ## Choosing the right strength The rule is: **assert the strongest property the contract actually guarantees.** - A repository method documented as returning results ordered by creation date: `shouldContainExactly`. If you weaken it to `InAnyOrder`, the ordering bug ships. - A method returning a `Set`, or results from a source with no defined iteration order: `shouldContainExactlyInAnyOrder`. Using `Exactly` here buys you a test that passes on your machine and fails in CI. - Only a couple of elements are part of this test's contract and the rest belong to other tests: `shouldContainAll`, with the test name saying so. A common anti-pattern is the weak pair: `result shouldHaveSize 3` plus `result shouldContain expected`. Together those pass for many wrong results. One `shouldContainExactly` or `shouldContainExactlyInAnyOrder` is both shorter and strictly stronger. ## Vararg and collection forms All of these come in both forms — `shouldContainExactly(1, 2, 3)` with varargs and `shouldContainExactly(listOf(1, 2, 3))` as an infix call with a collection. The vararg form reads better inline; the infix form is handy when the expected value is already a variable. ## Interview-ready summary Exactly = order + size + duplicates. InAnyOrder = size + duplicates, no order (multiset, *not* set). All = subset, extras and duplicates ignored. Choose by what the code under test actually guarantees, and never weaken a matcher just to stop a test flaking — that is a signal the ordering is undefined, which is information the assertion should record honestly.

  • Does shouldContainExactlyInAnyOrder treat the collections as sets?
    No, it treats them as multisets. Sizes must match and each element must appear the same number of times on both sides, so `listOf(1, 1, 2)` fails against `listOf(1, 2, 2)` even though the distinct values are identical. To get true set semantics you have to convert both sides with `toSet()` first, which means giving up on duplicate detection.
  • A test asserts result shouldHaveSize 3 and result shouldContain "b". What is wrong with that?
    It is a weak pair: any three-element result containing "b" passes, so wrong elements, wrong order and duplicated entries all go undetected. A single `shouldContainExactly` or `shouldContainExactlyInAnyOrder` is shorter, strictly stronger and gives a better failure report naming missing and unexpected elements.

saying these in an interview costs you the question

  • Describing shouldContainExactlyInAnyOrder as a set comparison
  • Believing shouldContainAll enforces size or ordering
  • Downgrading Exactly to InAnyOrder to stop a flaky test without checking whether order is part of the contract
  • Asserting size plus a single contains and calling it complete
  • Thinking shouldContainExactly ignores duplicates

context

open as a page

In Kotest, what equality does the shouldBe matcher actually apply — how does it compare arrays, data classes and collections, and where does that differ from a plain Kotlin == check?

level: middleimportance: must knowfreq 45%

basics

~20 s

Kotest's shouldBe does not simply call equals. It dispatches on runtime type first: arrays are compared by contents rather than by reference, iterables and maps element-wise with a positional diff, data classes with per-field failure reporting. Everything else falls back to equals.

open as a page

Kotest's MatcherResult carries a passed flag plus two messages — failureMessage and negatedFailureMessage. Under what circumstances does each message reach the developer, and what breaks if you write them carelessly?

level: middleimportance: must knowfreq 40%

basics

~20 s

failureMessage is shown when the matcher is used positively (should / a shouldX extension) and does not pass. negatedFailureMessage is shown when it is used negatively (shouldNot) and it does pass. Copying the positive text into the negated slot makes failures report the opposite of reality.

open as a page

Kotest's shouldThrow<T> block returns a value. What is that value, how do you assert on it, and when do you need the shouldThrowUnit variant instead?

level: middleimportance: must knowfreq 52%

basics

~20 s

It returns the caught exception, already typed as T. Bind it and assert on message, cause or custom fields. Use shouldThrowUnit<T> when the block's last expression is Unit, because plain shouldThrow expects a block returning Any?.

open as a page

How does Kotest's assertSoftly actually aggregate failures — what does it collect, what does it report, and which failures inside the block does it NOT aggregate?

level: seniorimportance: must knowfreq 46%

basics

~20 s

Inside the block Kotest switches its error collector to soft mode, so matcher failures are recorded instead of thrown; at the end it throws one MultiAssertionError listing them all. Only failures routed through Kotest's collector are aggregated — non-assertion exceptions abort the block immediately.

open as a page

What is the difference between Kotest's shouldBeSameInstanceAs and shouldBe, and when would you assert the former in a real test?

level: juniorimportance: should knowfreq 30%

basics

~20 s

shouldBe asserts the two values compare equal (equals, or Kotest's type-aware comparison). shouldBeSameInstanceAs asserts they are literally the same object in memory — a reference check like ===. Two equal data class instances pass the first and fail the second.

open as a page

In Kotest, does the string matcher shouldMatch require the whole string to match the regex or just part of it — and how does that differ from shouldContain with a Regex, shouldStartWith and shouldContainIgnoringCase?

level: middleimportance: should knowfreq 32%

basics

~20 s

Kotest's shouldMatch requires the entire string to match the regex — it is a full match, not a search. To assert that a pattern occurs somewhere inside a string, pass a Regex to shouldContain, which looks for a match anywhere. shouldStartWith/shouldEndWith are literal, case-sensitive; shouldContainIgnoringCase is a case-insensitive substring check.

open as a page

Kotest offers both shouldBeInstanceOf<T>() and shouldBeTypeOf<T>(). What is the difference in what they assert, and what can neither of them check?

level: middleimportance: should knowfreq 38%

basics

~20 s

shouldBeInstanceOf<T>() passes when the value is a T or any subtype. shouldBeTypeOf<T>() requires the runtime class to be exactly T, so a subclass fails. Neither can check generic type arguments, because those are erased at runtime.

open as a page

Kotest's Matcher type exposes an `invert()` function, and any matcher can also be used through `shouldNot`. How does negation actually work at the MatcherResult level, and when do you need invert() rather than just calling shouldNot?

level: middleimportance: should knowfreq 24%

basics

~20 s

shouldNot negates at the call site: it fails when the matcher passes, printing negatedFailureMessage. invert() negates the matcher itself, returning a new Matcher whose passed flag is flipped and whose two messages are swapped — needed when you must hand a negated matcher to something that takes a Matcher value.

open as a page

What do Kotest's withClue and asClue add to a failing assertion, and what happens when several of them are nested around the same matcher?

level: middleimportance: should knowfreq 38%

basics

~20 s

They attach context to any failure raised inside their block: the clue text is prepended to the matcher's own message. Nested clues stack — every active clue appears, outermost first — and asClue is the form that uses the receiver object itself as the clue.

open as a page

A Kotest assertion using shouldContainExactly on a repository result passes locally and fails intermittently in CI. How do you decide between tightening, weakening or restructuring the assertion?

level: seniorimportance: should knowfreq 26%

basics

~20 s

Decide from the contract, not from the flake. If ordering is guaranteed (an explicit ORDER BY), the flake is a real bug — keep shouldContainExactly and fix the source. If ordering is genuinely undefined, switch to shouldContainExactlyInAnyOrder, which still pins size and duplicates. Never fall back to size-plus-contains.

open as a page

A Kotest assertion using shouldBe fails even though the values look right — for example two BigDecimals, two computed doubles, or two entities differing only in a generated id and timestamp. Which Kotest matchers do you reach for instead, and why?

level: seniorimportance: should knowfreq 30%

basics

~20 s

shouldBe ultimately uses equals, which is the wrong test in those cases. Use plusOrMinus for floating point tolerance, shouldBeEqualComparingTo for BigDecimal (compareTo ignores scale), and the reflective matchers shouldBeEqualToIgnoringFields or shouldBeEqualToUsingFields for objects with noisy fields.

open as a page

Kotest's shouldBeInstanceOf<T>() and shouldNotBeNull() return a value rather than Unit. What do they return, how does that interact with Kotlin smart casts, and why does it matter for test readability?

level: seniorimportance: should knowfreq 28%

basics

~20 s

They return the value narrowed to the asserted type — T for shouldBeInstanceOf, the non-null T for shouldNotBeNull — and they carry Kotlin contracts so the original variable also smart-casts afterwards. That replaces as casts and !! with assertions that fail with real messages.

open as a page

When you combine two Kotest matchers with `and` or `or`, how is the combined MatcherResult produced — which sub-matcher's failure message do you actually see, and what does the combination cost you in evaluation and message quality?

level: seniorimportance: should knowfreq 28%

basics

~20 s

Both combinators short-circuit and return one sub-result verbatim. and returns the first failure, else the second result; or returns the first pass, else the last failure. So you see one side's message, never an assembled "A and B" sentence — and the value may be tested twice.

open as a page

You already have a Kotest Matcher<String> and want to apply the same rule to a field of a domain object, and separately to a nullable value. What does Matcher's contramap give you, and what problem does Kotest's neverNullMatcher helper solve?

level: seniorimportance: should knowfreq 22%

basics

~20 s

contramap adapts a matcher's input type: Matcher<String>.contramap { o: Order -> o.ref } yields a Matcher<Order> that applies the same rule to the projected field. neverNullMatcher wraps a non-null test into a Matcher<T?> that fails with a message on null instead of throwing NPE.

open as a page

How does Kotest's shouldThrowExactly<T> differ from shouldThrow<T> in what it accepts, and what happens in either one when the block throws an AssertionError that is not the expected type?

level: seniorimportance: should knowfreq 34%

basics

~20 s

shouldThrow<T> accepts T or any subtype; shouldThrowExactly<T> requires the runtime class to be exactly T and fails on subtypes. If the block throws an AssertionError that isn't the expected type, Kotest rethrows it as-is so a nested assertion failure isn't masked as a wrong-exception report.

open as a page

How would you design a shared set of domain-specific Kotest matchers for a team — the naming and negation surface, how the assertions are exposed to test authors, and what keeps failure output good as the set grows?

level: principalimportance: should knowfreq 18%

basics

~20 s

One function per rule returning a Matcher<T>, named as a predicate phrase (beShippable()). Expose paired shouldX / shouldNotX extensions that delegate to should / shouldNot and return the receiver. Never throw from inside a matcher, keep matchers pure, and review failure wording like production copy.

open as a page

Kotest has shouldBeSorted plus matchers such as shouldBeSortedWith, shouldBeMonotonicallyIncreasing and shouldBeStrictlyIncreasing. What does each assert, and what are the traps in asserting sortedness?

level: middleimportance: nice to knowfreq 22%

basics

~20 s

shouldBeSorted checks natural ascending order on Comparable elements and allows equal neighbours. shouldBeSortedWith takes a custom comparator. Monotonically increasing allows ties; strictly increasing forbids them. Traps: empty and single-element collections pass trivially, and sortedness alone does not pin the contents.

open as a page

You own a large Kotlin test suite. How would you decide where Kotest's assertSoftly aggregation and withClue context are used, versus letting assertions fail fast?

level: principalimportance: nice to knowfreq 26%

basics

~20 s

Aggregate independent observations of one state (fields of a response, elements of a collection) and fail fast on preconditions and dependent assertions. Add clues wherever a failure could come from more than one input. Judge by whether one defect produces one report line.

open as a page