skip to content

You wrote flow.toList() and asserted an exact ordered list, but the test is flaky—sometimes the order differs. What likely causes this and how do you make collection deterministic?

level: seniorimportance: nice to knowfreq 33%

answer

  1. Sequential flow = deterministic order
  2. Flaky -> concurrency: flatMapMerge/merge/buffer/real dispatcher
  3. flatMapConcat preserves order; flatMapMerge doesn't
  4. Stay on StandardTestDispatcher for serialization
  5. Order-agnostic? assert toSet() or sorted()

basics

~20 s

A single sequential flow emits in a fixed order, so flakiness usually means concurrency: the flow uses flatMapMerge, merge, channels, or a real dispatcher that interleaves results. Use ordering-preserving operators or sort the result before asserting.

solid answer

~40 s

toList() preserves the emission order of the flow, and a plain sequential flow { } emits deterministically. Flaky ordering means the flow introduces concurrency: flatMapMerge/flattenMerge interleave inner flows, merge() races multiple sources, buffer()/channels add concurrent producers, or the work runs on Dispatchers.Default/IO so results arrive in nondeterministic real-thread order. Fixes: (1) use order-preserving operators—flatMapConcat instead of flatMapMerge, or map instead of merging; (2) keep everything on the single test dispatcher (StandardTestDispatcher) so virtual time serializes execution; (3) if order genuinely doesn't matter, assert as a set: assertEquals(expected.toSet(), result.toSet()) or sort both sides before assertEquals. Don't 'fix' flakiness with sleeps or retries. The root cause is asserting an ordered list against an unordered/concurrent emission.

code

kotlin · 6 lines
kotlin
@Test
fun orderAgnosticAssertion() = runTest {
    val result = merge(flowOf(1, 2), flowOf(3, 4)).toList()
    // order between the two sources is not guaranteed:
    assertEquals(setOf(1, 2, 3, 4), result.toSet())
}

go deeper

for a junior

Knows toList() keeps emission order and that a simple flow is deterministic.

for a middle

Identifies merge/flatMapMerge as concurrency and can assert via toSet() when order doesn't matter.

for a senior

Diagnoses real-dispatcher leakage, swaps to flatMapConcat/StandardTestDispatcher, and rejects sleep-based fixes.

for a principal

Drives conventions distinguishing order-significant vs order-agnostic flows and prevents concurrency leaks in shared code.

## Why a sequential flow is deterministic `toList()` returns emissions in the exact order the flow produced them. A plain `flow { emit(a); emit(b) }` is sequential—`a` always precedes `b`. So if `assertEquals(listOf(a, b), result)` is flaky, the flow is **not** purely sequential. ## Common sources of nondeterministic order - **`flatMapMerge` / `flattenMerge`**: run multiple inner flows concurrently and interleave their emissions. Two inner flows' items can arrive in either order. - **`merge(f1, f2)`**: emits from several flows as values arrive—an inherent race. - **`buffer()` / `channelFlow` / `produce`**: introduce a concurrent producer/consumer boundary. - **Real dispatchers** via `flowOn(Dispatchers.IO/Default)` or `withContext`: work runs on real threads outside the test scheduler, so completion order is timing-dependent. ## Making it deterministic ### 1. Use order-preserving operators ```kotlin // Nondeterministic order: sources.flatMapMerge { fetch(it) }.toList() // Deterministic—processes one inner flow fully before the next: sources.flatMapConcat { fetch(it) }.toList() ``` `flatMapConcat` keeps source order; `flatMapMerge` does not. ### 2. Stay on the test dispatcher Inject `StandardTestDispatcher(testScheduler)` so all coroutines share the virtual clock and are serialized by the scheduler instead of racing on real threads. Avoid `flowOn(Dispatchers.IO)` in code under test, or override it in the test. ### 3. Assert order-independently when order truly doesn't matter ```kotlin val result = concurrentFlow.toList() assertEquals(expected.toSet(), result.toSet()) // ignore order // or assertEquals(expected.sorted(), result.sorted()) // canonicalize order ``` ## Anti-patterns - Adding `delay()`/`Thread.sleep` to 'stabilize' order—masks the bug and slows tests. - Retrying the test until it passes. - Asserting an ordered `listOf(...)` against an intrinsically unordered (merged) stream. ## Decision guide - Order is a real requirement -> use concat/sequential operators and assert the ordered list. - Order is irrelevant -> assert as `Set` or sort both sides. ## Keywords `flatMapConcat` vs `flatMapMerge`, `merge`, `buffer`, `channelFlow`, `flowOn`, `StandardTestDispatcher`, `toSet()`, `sorted()`.

  • Between flatMapConcat and flatMapMerge, which preserves source order and why?
    flatMapConcat: it collects each inner flow to completion before starting the next, so emissions stay in source order. flatMapMerge runs inners concurrently and interleaves them.
  • Is adding a small delay a valid fix for ordering flakiness?
    No—it hides a real concurrency bug, can still fail under load, and (with virtual time) may not even change scheduling. Fix the operator/dispatcher or assert order-independently.

flatMapConcat is a single-file checkout line; flatMapMerge is several lanes finishing in whatever order—don't assert a fixed receipt order across lanes.

saying these in an interview costs you the question

  • Insisting toList() randomizes order by itself
  • Adding delay()/sleep to stabilize ordering
  • Asserting an ordered list against a merge() result
  • Not recognizing flatMapMerge as a concurrency source
  • Leaving Dispatchers.IO in code under test and blaming the assertion

context

resolving links…