A Kotest assertion using shouldContainExactly on a repository result passes locally and fails intermittently in CI. How do you decide between tightening, weakening or restructuring the assertion?

## The decision, not the fix This question is about judgement. Any candidate can make a flaky assertion green; the interviewer wants to hear you route through the contract first. ## Step 1: what does the code under test promise? Ask what defines the order of the result: - an explicit, total ordering in the query or in the code (`ORDER BY created_at, id`) — order **is** contractual; - an ordering with ties and no tie-breaker (`ORDER BY score` where scores collide) — order is *partially* contractual; - no ordering clause at all, a `Set`, a `HashMap` traversal, results assembled from concurrent work — order is **not** contractual and any stability you observed locally was luck. Only the third case justifies weakening. In the first, an intermittent failure is a genuine defect: the code promises an order it does not deliver, and users would see rows shuffle between page loads. Weakening the test there converts a caught bug into a shipped one. ## Step 2: assert exactly what is contractual Map each case onto a matcher. **Fully ordered contract.** ```kotlin results shouldContainExactly listOf(rowA, rowB, rowC) ``` Keep it. Fix the query — usually by adding a deterministic tie-breaker such as the primary key to the ordering. **No ordering contract.** ```kotlin results shouldContainExactlyInAnyOrder listOf(rowA, rowB, rowC) ``` This stays strong on the axes that still matter: same size, same elements, same duplicate counts. It is a one-notch weakening, not a surrender. **Partial ordering — ordered by a key with ties.** ```kotlin results shouldContainExactlyInAnyOrder expectedRows // contents results.shouldBeSortedWith(compareByDescending { it.score }) // the guaranteed order ``` Two assertions, each pinning one real property. This is the answer that distinguishes a senior candidate: it neither over-specifies (demanding an arrangement the system never promised) nor under-specifies. ## Step 3: the things not to do **Sorting the actual value to force a match.** ```kotlin results.sortedBy { it.id } shouldContainExactly expected.sortedBy { it.id } ``` It works, but the test now silently means "any order" while looking like an ordered assertion — the reader has to notice the `sortedBy` to understand the contract. `shouldContainExactlyInAnyOrder` states it in the matcher name. **Falling back to a weak pair.** ```kotlin results shouldHaveSize 3 results shouldContain rowB ``` This passes for any three-element result containing `rowB` — wrong elements, duplicates, missing rows all go undetected. It is the most common way a flaky test becomes a useless test. **Retrying or ignoring.** Marking the test flaky-tolerant hides an ordering defect and, worse, trains the team to ignore that class of failure. ## Step 4: consider the fixture, not just the matcher Sometimes the right fix is upstream of the assertion entirely: - If nondeterminism comes from shared state between tests (a database not reset, a static cache), the order flake is a symptom of test isolation, and no matcher change fixes it. - If the result is genuinely unordered but the *test* would be clearer over a canonical form, map to a projection first: `results.map { it.id } shouldContainExactlyInAnyOrder listOf(1, 2, 3)`. Comparing small projections gives sharper failure output than comparing big entities. ## What good looks like in the answer 1. Treat the flake as a question about the contract, not an inconvenience. 2. Name `shouldContainExactly` versus `shouldContainExactlyInAnyOrder` precisely, including that the latter still enforces size and duplicate counts. 3. Offer the split assertion — contents in any order plus `shouldBeSortedWith` for the guaranteed ordering key — for the partial-order case. 4. Explicitly reject the size-plus-contains fallback and in-test sorting as intent-hiding. 5. Mention that the flake may not be about ordering at all, but about test isolation.