skip to content

In an async-heavy test suite, when would you rely on MockK's verify(timeout = ...) versus building deterministic synchronization — for example counting down a latch inside an answers block, or injecting the executor or dispatcher so the work runs on the test thread? How do you keep such a suite from becoming flaky or slow?

level: principalimportance: nice to knowfreq 18%

answer

  1. inject seam > latch barrier > timeout poll > sleep (never)
  2. latch in answers { } = defensible "done" moment
  3. absence assertions only after a barrier
  4. generous deadlines: only failures pay
  5. many waiting tests = missing execution seam

basics

~20 s

Prefer determinism where you own the seam: inject the executor or dispatcher so the work is synchronous, or count a latch down inside an answers block. Use timeout verification at edges you cannot control. Never sleep, and set timeouts generously since only failures pay.

solid answer

~60 s

My default is to remove the concurrency from the test rather than wait it out: if the component takes its executor or dispatcher as a dependency, the test supplies one that runs work on the calling thread and every assertion becomes an ordinary, instant verify. That is the cheapest and least flaky option, and it also documents the seam. Where I cannot inject — third-party callbacks, framework-owned threads, legacy code — I use a MockK stub as the synchronisation point: `every { listener.onDone() } answers { latch.countDown() }`, await the latch, then assert. That converts an unknown wait into an explicit barrier and still gives me exact-count and ordering assertions afterwards. `verify(timeout = ...)` is the low-ceremony version of the same barrier, and I use it freely for simple "eventually called" assertions. Its limits are what drive the policy: it cannot prove absence, the shorthand ordered verifications take no timeout, and every failure costs its full deadline. So timeouts stay generous, few, and always paired with a positive expectation.

code

kotlin · 8 lines
kotlin
val done = CountDownLatch(1)
every { listener.onFinished(any()) } answers { done.countDown() }

uploader.upload(file)

assertTrue(done.await(5, TimeUnit.SECONDS), "upload never finished")
verify(exactly = 1) { listener.onFinished(file) }
verify(exactly = 0) { auditLog.record(any()) }

go deeper

for a junior

Know the ranking: make it synchronous if you can, otherwise wait with a bounded MockK timeout, and never sleep.

for a middle

Be able to write the latch-in-an-answers-block barrier and explain what it buys over a plain timeout verify.

for a senior

Argue the tradeoff per test based on who owns the code, and set timeout values from CI reality rather than laptop timings.

for a principal

Turn it into policy plus a design signal: the number of tests that need waiting measures how much of the codebase creates its own concurrency, and that is the thing to drive down.

## Framing the decision There are three ways a test can cope with work that happens off the test thread: make it not happen off the test thread (inject the execution seam), wait for an explicit signal (a latch or barrier driven from a stub), or poll for the outcome (MockK's `verify(timeout = ...)`). They are ordered from most deterministic to least, and from most intrusive on production design to least. The engineering call is where on that spectrum a given test should sit, and the answer usually follows from who owns the code. ## Option 1 — inject the seam (default when you own the code) If a component accepts its `Executor`, `CoroutineDispatcher` or scope as a constructor dependency, the test can hand it one that executes work on the calling thread. Concurrency disappears from the test entirely: the call under test returns only after the collaborator was touched, every assertion is a plain `verify`, exact counts and ordering are meaningful, and absence assertions are real. Run time is zero-cost and there is nothing to flake. The price is a design constraint on production code — the class must not create its own threads internally. In practice that constraint is worth paying for on its own merits, because it is also what makes timeouts, shutdown and back-pressure configurable in production. When I review a class that needs timeout verification to be testable, my first question is whether it should be taking its executor as a parameter. ## Option 2 — an explicit barrier built from a stub When you cannot control where the work runs, you can still control what happens *when it lands*, because the collaborator is a mock: ``` val done = CountDownLatch(1) every { listener.onFinished(any()) } answers { done.countDown() } uploader.upload(file) assertTrue(done.await(5, TimeUnit.SECONDS), "upload never finished") verify(exactly = 1) { listener.onFinished(file) } verify(exactly = 0) { auditLog.record(any()) } ``` This is strictly more capable than polling: the latch establishes a defensible "the work is finished" moment, after which exact counts, ordering assertions and negative assertions are all legitimate. The failure message is also yours to write, which is worth more than it sounds when a test fails once a fortnight on CI. The cost is a few lines of ceremony and the discipline to count the latch down in the right stub. ## Option 3 — timeout verification `verify(timeout = ...)` is the same barrier with no ceremony and no production seam. For the common case — "this collaborator eventually gets called" — it is the right tool and I would not push back on it in review. Its limitations define where it stops: it cannot assert absence (an already-true verification returns immediately), `verifyOrder`/`verifySequence` take no timeout, and it tells you nothing about *why* a call failed to arrive. ## Keeping the suite fast and stable A few rules that hold up across large suites: * **Never sleep.** A sleep pays unconditionally and still asserts at one guessed instant. There is no case where it beats a bounded wait. * **Be generous with deadlines.** Success returns early, so a five-second timeout on a green test costs milliseconds. Tightening timeouts optimises the fast path and buys flakiness with the saving. Loaded CI agents are orders of magnitude slower than a developer laptop at thread scheduling. * **Budget the failure cost.** Deadlines are only paid by failures, but a broken build with fifty waiting tests pays fifty deadlines serially. That, not green-run speed, is the argument for keeping the number of waiting tests small. * **Pair every wait with a positive expectation.** Absence assertions must sit *after* a barrier, never carry a timeout of their own. * **Watch for confined execution.** A wait that blocks the only thread able to run the pending work fails deterministically at the full deadline. Its signature is a failure duration that exactly equals the timeout, on every run. * **Capture worker failures.** Exceptions on background threads do not reach the test, so an uncaught-exception handler that records failures turns "no matching calls" into the real cause. ## The policy I would write down Inject the execution seam for code we own; use a latch-in-a-stub barrier for edges we do not; allow `verify(timeout = ...)` for simple eventual-call assertions with deadlines measured in seconds; ban sleeps outright; and treat a timeout on a negative assertion as a review defect. Then track how many tests need waiting at all — a growing count is a design signal about missing seams, not a testing problem.

  • A teammate wants to shorten every timeout from 5s to 200ms to speed up the suite. What do you say?
    That green runs already return in milliseconds, because the wait ends as soon as the call is recorded — so the change saves nothing on the fast path. What it buys is flakiness, since 200 ms is well inside normal thread-scheduling jitter on a loaded CI agent. If suite duration is the real concern, the lever is reducing how many tests need to wait at all, by injecting execution seams.
  • When is a latch-based barrier worth the extra lines over a plain timeout verify?
    Whenever the assertions after the wait need a defensible completion point: exact counts, ordering, or asserting that something did not happen. The latch gives you a moment in time you can reason about, and a failure message you control. A timeout verify only tells you that one expected call eventually arrived.

saying these in an interview costs you the question

  • Treating Thread.sleep and verify(timeout=) as interchangeable
  • Cutting timeouts to make the suite faster, when green runs already return early
  • Putting a timeout on a negative assertion instead of a barrier before it
  • Refusing to inject the executor/dispatcher and calling the resulting waits unavoidable
  • Assuming a flaky async test is always the test's fault rather than a missing design seam

context