How does MockK's verify(timeout = ...) combine with count modes such as exactly, atLeast and atMost, and why does adding a timeout to a "never happened" assertion (exactly = 0, wasNot Called, or inverse = true) give you false confidence?
answer
- retry while failing, return on first success
- atLeast waits; atMost/exactly=0 pass instantly
- exactly = n: lower bound only, surplus unseen
- wasNot Called + timeout = decoration
- wait for a positive signal, then assert absence
basics
~20 sMockK retries while the verification fails and stops at the first success. So atLeast waits for the nth call, but any already-satisfied check — exactly = 0, wasNot Called, inverse, or an atMost that is currently under the bound — passes on the first attempt and never waits at all.
solid answer
~50 sThe timeout wraps the same verification you would otherwise run once: MockK keeps re-evaluating it and returns the moment it is satisfied. That makes it genuinely useful for lower-bound assertions — `verify(timeout = 2000, atLeast = 3) { mock.tick() }` waits until the third call lands. It is useless, and actively misleading, for anything that is true at the start. `exactly = 0`, `wasNot Called`, `inverse = true` and a not-yet-exceeded `atMost` are all satisfied on the very first evaluation, so `verify` returns immediately and asserts nothing about the future. The test reads as "we waited two seconds and it never happened" but actually checked once, at time zero. `exactly = n` is a half-measure: it waits until the count reaches n, then returns — an n+1th call arriving later is never observed. To assert absence, first wait for a positive signal you *do* expect, then assert the negative.
code
kotlin · 9 lines// waits until the third call arrives
verify(timeout = 2000, atLeast = 3) { collector.publish(any()) }
// satisfied at time zero -> returns immediately, proves nothing
verify(timeout = 2000, exactly = 0) { auditLog.record(any()) }
// correct: barrier first, then the negative check
verify(timeout = 2000) { listener.onFinished(any()) }
verify(exactly = 0) { auditLog.record(any()) }go deeper
Know that a timeout waits for something to happen, so it fits "was called" and not "was never called".
State the retry-while-failing rule and apply it to atLeast, atMost and exactly; recognise that already-true assertions return instantly.
Diagnose the false-confidence case in review, explain why exactly = n only enforces its lower bound, and prescribe the barrier-then-assert-absence pattern.
Treat decorative timeouts on negative assertions as a suite-wide defect class, and push for injectable execution seams where absence assertions actually matter.
## The rule that explains everything MockK's timeout verification is a retry loop around an ordinary verification: while the verification is unsatisfied and the deadline has not passed, re-evaluate; on success, return. Every interaction with count modes falls out of that single rule — ask "is this assertion true right now, before the async work happens?" If yes, the timeout adds nothing at all. ## Lower bounds: where timeout earns its keep `atLeast = n` starts out false when fewer than n calls have been recorded, so the loop keeps retrying and the verification becomes true exactly when the nth call arrives. This is the natural pairing: ``` verify(timeout = 2000, atLeast = 3) { collector.publish(any()) } ``` The default `atLeast = 1` is the same shape, which is why the plain `verify(timeout = ...) { ... }` form behaves so intuitively. ## Upper bounds and exact counts: partial at best `atMost = n` is satisfied whenever the recorded count is at or below n — which it usually is at time zero. The loop exits on the first evaluation and you have proved nothing; the excess calls you were worried about may all still be queued. `exactly = n` is a genuine half-measure. It is false while the count is below n, so the loop does wait for the nth call. But the moment the count hits n the verification succeeds and `verify` returns. A surplus n+1th call that arrives a millisecond later is never seen. So `exactly` with a timeout enforces the lower bound reliably and the upper bound only by luck of timing. If the upper bound matters, you need a different mechanism entirely: wait for a deterministic end-of-work signal, then assert the exact count with a plain, non-timeout verify. ## Negative assertions: the false-confidence trap The worst case is asserting that something never happens: ``` // looks careful, proves nothing verify(timeout = 2000, exactly = 0) { auditLog.record(any()) } verify(timeout = 2000) { auditLog wasNot Called } verify(timeout = 2000, inverse = true) { auditLog.record(any()) } ``` All three are true before the background work has had a chance to do anything, so all three succeed on the first evaluation and return immediately. The test costs nothing, always passes locally, and will pass just as happily on the day the code starts making the forbidden call — because the call arrives after the assertion already returned. The number `2000` is pure decoration; reviewers read it as patience the test does not have. ## The correct shape for asserting absence in async code Absence is only meaningful relative to a point in time you can defend. The reliable pattern is to establish a barrier out of something positive that you *do* expect, and then assert the negative once, without a timeout: ``` // 1. wait for a real completion signal verify(timeout = 2000) { listener.onFinished(any()) } // 2. now the pipeline is done; assert the negative verify(exactly = 0) { auditLog.record(any()) } ``` The barrier does the waiting; the negative assertion is then evaluated at a defensible moment. If there is no observable completion signal to wait for, the honest answer is that the production code lacks a testable seam — inject the executor or dispatcher so the work is deterministic, rather than dressing the race up as a timeout. ## Cost asymmetry to keep in mind Because the loop exits on success, timeouts are free on the paths where they work (lower bounds that eventually hold) and free on the paths where they are useless (already-true negatives). The only case that pays the full deadline is a genuine failure. That asymmetry is why generous timeouts are fine, and also why a suite full of decorative timeouts on negative assertions never shows up as slow — the uselessness is invisible in the run time.
- So is exactly = n with a timeout ever acceptable?It is acceptable when n is really a lower bound you care about and a surplus call would be caught elsewhere. It reliably waits for the nth call, but returns the instant the count matches, so it cannot see an extra call arriving later. If "no more than n" is the actual requirement, wait on a completion signal and then assert the count without a timeout.
- How would you assert that a fire-and-forget path made no call, when there is no completion callback to wait for?Do not try to prove it with a timeout. Give the code a seam: inject the executor or dispatcher so the test can run the work on the calling thread, or expose an idle/awaitable handle. Once execution is deterministic, a plain verify(exactly = 0) is a real assertion rather than a race you happen to win.
saying these in an interview costs you the question
- "The timeout makes wasNot Called wait two seconds to be sure"
- Believing exactly = 0 with a timeout proves the call never arrives
- Thinking atMost with a timeout catches surplus calls
- Assuming MockK buffers all calls made during the timeout window before evaluating once
- Adding timeouts everywhere "for safety" without asking whether the assertion is already true