In Kotest, what exactly is the difference between shouldContainExactly, shouldContainExactlyInAnyOrder and shouldContainAll — particularly with respect to ordering, size and duplicate elements?
answer
- Exactly = positional
- InAnyOrder = multiset, not set
- Duplicate counts still matter in InAnyOrder
- All = subset, extras allowed
- Weak pair: size + contains
basics
~20 sshouldContainExactly requires the same elements in the same order and the same size. shouldContainExactlyInAnyOrder requires the same multiset — same size and same duplicate counts, order irrelevant. shouldContainAll is only a subset check: extras and duplicate mismatches are ignored.
solid answer
~50 sThey form a strength ladder. - `shouldContainExactly(1, 2, 2)` — positional comparison. Same size, same elements, same order, duplicates counted. `listOf(2, 1)` fails against `listOf(1, 2)`. - `shouldContainExactlyInAnyOrder(1, 2, 2)` — multiset comparison. Order is irrelevant, but size and multiplicity are not: `listOf(1, 1, 2)` does **not** satisfy `shouldContainExactlyInAnyOrder(1, 2, 2)`. This is the matcher people wrongly describe as a set comparison. - `shouldContainAll(1, 2)` — containment only. The actual collection may contain extra elements and any number of duplicates; it just has to include each named element at least once. - `shouldContain(1)` — single-element membership. Pick the strongest one the contract justifies. Use `Exactly` when order is part of the API contract (a sorted query, a pipeline output), `ExactlyInAnyOrder` when the source has no defined order, and `All` only when you deliberately want a partial assertion.
code
kotlin · 7 lineslistOf(1, 2, 3) shouldContainExactly listOf(1, 2, 3) // pass
listOf(1, 2, 3) shouldContainExactly listOf(3, 2, 1) // fail: order
listOf(1, 2) shouldContainExactlyInAnyOrder listOf(2, 1) // pass
listOf(1, 1, 2) shouldContainExactlyInAnyOrder listOf(1, 2, 2) // fail: duplicate counts
listOf(1, 2, 3, 4) shouldContainAll listOf(1, 3) // pass: subset onlygo deeper
Recall the three-way split: exact order, same elements any order, and subset containment.
Nail the multiset semantics of InAnyOrder with a duplicate example, and explain that Exactly also pins size and order.
Frame it as assertion strength: pick the strongest matcher the contract guarantees, and treat a flaking Exactly as evidence the order is undefined rather than as a reason to weaken the test.
Discuss it as a suite-wide convention — undefined ordering should be expressed once in the API contract and reflected consistently in the assertions, not rediscovered per test.
## The ladder Kotest's collection matchers differ along three axes: does order matter, does size matter, do duplicates matter. Placing each matcher on those axes is the whole question. | Matcher | Order | Size/extras | Duplicates | |---|---|---|---| | `shouldContainExactly` | must match | must match | counted, positionally | | `shouldContainExactlyInAnyOrder` | irrelevant | must match | counted | | `shouldContainAll` | irrelevant | extras allowed | ignored | | `shouldContain` | irrelevant | extras allowed | ignored (single element) | ## shouldContainExactly This is a full positional comparison — conceptually the same statement as `shouldBe` on two lists, but expressed as a collection assertion and reported as one. Same length, same elements, same order. ```kotlin listOf(1, 2, 3) shouldContainExactly listOf(1, 2, 3) // passes listOf(1, 2, 3) shouldContainExactly listOf(3, 2, 1) // fails: order listOf(1, 2) shouldContainExactly listOf(1, 2, 2) // fails: size ``` When it fails, Kotest reports the divergence — which elements are missing from the actual collection and which are present but unexpected — rather than just printing two lists and leaving you to diff them. ## shouldContainExactlyInAnyOrder The most misunderstood one. It is a **multiset** (bag) comparison, not a set comparison. Two collections match when they have the same size and every element occurs the same number of times in each. ```kotlin listOf(1, 2) shouldContainExactlyInAnyOrder listOf(2, 1) // passes listOf(1, 1, 2) shouldContainExactlyInAnyOrder listOf(1, 2, 2) // FAILS listOf(1, 1, 2) shouldContainExactlyInAnyOrder listOf(1, 2) // FAILS (size) ``` The second example is the one candidates get wrong: both sides contain exactly the values {1, 2}, so anyone thinking "set comparison" expects a pass. Duplicate counts differ, so it fails. That is a feature — a de-duplication bug in the code under test should not slip past an "any order" assertion. If you truly want set semantics, convert explicitly: `actual.toSet() shouldContainExactlyInAnyOrder expected.toSet()`, and be aware you have just stopped testing duplicates at all. ## shouldContainAll A containment/subset assertion. Every named element must be present at least once; the actual collection may contain anything else besides. ```kotlin listOf(1, 2, 3, 4) shouldContainAll listOf(1, 3) // passes listOf(1, 1, 1) shouldContainAll listOf(1) // passes ``` This is a genuinely weak assertion, and that is sometimes right — asserting that an audit log contains two specific events without pinning the rest. But it is often used as a way to make a flaky test green, and then the test no longer detects extra, duplicated or wrongly ordered output. ## Choosing the right strength The rule is: **assert the strongest property the contract actually guarantees.** - A repository method documented as returning results ordered by creation date: `shouldContainExactly`. If you weaken it to `InAnyOrder`, the ordering bug ships. - A method returning a `Set`, or results from a source with no defined iteration order: `shouldContainExactlyInAnyOrder`. Using `Exactly` here buys you a test that passes on your machine and fails in CI. - Only a couple of elements are part of this test's contract and the rest belong to other tests: `shouldContainAll`, with the test name saying so. A common anti-pattern is the weak pair: `result shouldHaveSize 3` plus `result shouldContain expected`. Together those pass for many wrong results. One `shouldContainExactly` or `shouldContainExactlyInAnyOrder` is both shorter and strictly stronger. ## Vararg and collection forms All of these come in both forms — `shouldContainExactly(1, 2, 3)` with varargs and `shouldContainExactly(listOf(1, 2, 3))` as an infix call with a collection. The vararg form reads better inline; the infix form is handy when the expected value is already a variable. ## Interview-ready summary Exactly = order + size + duplicates. InAnyOrder = size + duplicates, no order (multiset, *not* set). All = subset, extras and duplicates ignored. Choose by what the code under test actually guarantees, and never weaken a matcher just to stop a test flaking — that is a signal the ordering is undefined, which is information the assertion should record honestly.
- Does shouldContainExactlyInAnyOrder treat the collections as sets?No, it treats them as multisets. Sizes must match and each element must appear the same number of times on both sides, so `listOf(1, 1, 2)` fails against `listOf(1, 2, 2)` even though the distinct values are identical. To get true set semantics you have to convert both sides with `toSet()` first, which means giving up on duplicate detection.
- A test asserts result shouldHaveSize 3 and result shouldContain "b". What is wrong with that?It is a weak pair: any three-element result containing "b" passes, so wrong elements, wrong order and duplicated entries all go undetected. A single `shouldContainExactly` or `shouldContainExactlyInAnyOrder` is shorter, strictly stronger and gives a better failure report naming missing and unexpected elements.
saying these in an interview costs you the question
- Describing shouldContainExactlyInAnyOrder as a set comparison
- Believing shouldContainAll enforces size or ordering
- Downgrading Exactly to InAnyOrder to stop a flaky test without checking whether order is part of the contract
- Asserting size plus a single contains and calling it complete
- Thinking shouldContainExactly ignores duplicates