Across a large suite, when is waiting inside a mock verification the right tool at all, versus a sign the production code should expose a seam that makes the test deterministic? How would you set that policy for a team?
answer
- would it be interesting on an infinitely fast machine?
- inject Executor / Clock / scheduler
- timeout generous, after short and rare
- absence assertions dominate wall clock
- no sleeps, no blanket retries, quarantine flakes
basics
~20 sTimed verification is legitimate where asynchrony is the behaviour under test — typically integration-level boundaries. In unit tests it usually means the class hides its executor. Policy: inject execution seams, ban sleeps, allow generous timeouts at boundaries, keep full-wait afters rare and budgeted.
solid answer
~50 sI split tests into two categories. Asynchrony as an implementation detail: the class under test submits work to an executor it created itself. The fix is a seam — inject the Executor and pass Runnable::run, or a synchronous task executor — so the test is fully deterministic and no timed verification exists. Most unit tests are here, and timed verification here is a smell. Asynchrony as the contract: message listeners, schedulers, retry/backoff, batching, debouncing. Timing is what you are testing, so waiting is legitimate. There I use generous timeouts (upper bounds, free on success), latches for exact counts, and short after windows for absence, ideally replaced by a completion signal. Policy artefacts: no Thread.sleep in tests, enforced by a lint rule; named constants for wait budgets rather than magic numbers; every test owns and shuts down its executors; no blanket CI retries — flakes get quarantined with an owner; track aggregate wall-clock time spent waiting as a suite metric so absence assertions stay visible.
go deeper
Focus on the concrete rule: prefer making the test synchronous by injecting the executor; use a wait only when the code really is asynchronous.
Contrast incidental vs contractual asynchrony and name the seams (Executor, Clock, test scheduler) plus correct sizing of timeout vs after.
Argue the tradeoffs, including what determinism costs in lost race coverage, and describe how you would migrate an existing flaky suite.
Deliver it as enforceable policy with mechanisms (lint rules, named budgets, ownership of flakes, suite wait-time metric) and an explicit layering strategy that keeps some real concurrency coverage.
## The decision that matters Every timed mock verification is a bet on machine speed. The engineering question is not which API to use but whether the test should be making that bet at all. Sort each case by asking: if the runtime were infinitely fast, would this test still be interesting? If yes, the asynchrony is incidental and should be designed out of the test. If the interesting part is exactly 'this happens later, in this order, at most this often', the asynchrony is the subject and waiting is honest. ## Designing the asynchrony out The standard seam is to stop constructing execution strategy inside a class. Inject an Executor, a ScheduledExecutorService, a Clock, or a framework task executor. In tests, pass a direct executor (Runnable::run) so submitted work completes before submit returns, a deterministic scheduler that advances virtual time on command, or a fixed Clock. Reactive and coroutine stacks have equivalents (test schedulers, virtual time). The payoff is large: the test asserts the same behaviour with plain verify, runs in microseconds, cannot flake, and fails with a real stack trace instead of a wait timeout. The cost is one more constructor parameter and the discipline to keep production wiring in one place. ## Where waiting earns its place Some contracts cannot be collapsed. A broker listener genuinely delivers on another thread; a retry policy is defined in wall-clock terms; a batcher flushes on a timer; a component under test wraps a third-party library that owns its threads. For those, timed verification is the right instrument, but sized deliberately: timeouts are upper bounds and should be generous, because a passing run pays only the real latency, while a tight budget converts every slow runner into a red build. Full-wait after windows are the opposite — every millisecond is paid on every run — so they should be short, rare, and justified. ## Absence is the expensive claim Asserting that something did not happen is the only assertion that must burn time by construction, and its confidence is bounded by the window. At suite scale these dominate the wall clock and quietly degrade as people shorten windows to speed up CI. Push teams to convert absence assertions into positive terminal signals: drive the pipeline to a completion barrier, then use verifyNoInteractions or verifyNoMoreInteractions with no waiting at all. ## Making the policy stick A policy only works if it is mechanical. Ban Thread.sleep in test sources with a static-analysis rule. Require wait budgets to be named constants so their aggregate is greppable and tunable in one place. Require every test that starts threads to shut them down, since leaked pools cause cross-test flakes that look like timing bugs. Refuse blanket retry-on-failure in CI: it hides the failure class where the async task threw and was swallowed, which is a product bug. Instead quarantine flakes with a named owner and a deadline. ## Reviewing the tradeoff There is a real counter-argument to full determinism: a direct executor changes the concurrency shape, so it will not catch races, visibility bugs, or ordering assumptions that only appear with real threads. The answer is layering — deterministic unit tests for logic, plus a small number of genuinely concurrent tests (and, where it pays, stress or property tests) at the integration layer where the timed verification lives. The policy is not 'never wait'; it is 'wait in few, well-chosen places, generously, and never by sleeping'.
- What do you lose by running everything through a direct executor in tests?You lose the concurrency itself: races, memory-visibility bugs, deadlocks and ordering assumptions cannot appear when the work runs on the calling thread. That is acceptable for logic-focused unit tests but must be compensated with a thin layer of genuinely concurrent integration tests, and where the risk is high, stress or property-based testing around the real executor.
- How do you keep a team from silently trimming wait budgets to speed up CI?Make the budgets named constants in one place and review changes to them like production changes, and measure the suite's total time spent waiting as a tracked metric. If that number grows, the answer is fewer waiting tests or better seams, not smaller numbers — shrinking windows weakens every absence assertion without anyone noticing.
saying these in an interview costs you the question
- Treating timed verification as the normal way to test any async code
- Claiming a direct executor in tests proves the code is thread-safe
- Allowing Thread.sleep as long as it is 'only a small one'
- Solving flakiness with CI retries instead of seams
- Ignoring the cumulative wall clock of full-wait absence assertions