How do you use unittest.mock side_effect to test a retry path that times out twice then succeeds?

Intermittent failures are the hardest thing to test honestly, because the property you want — "it recovers" — depends on a sequence of outcomes rather than on any single one. `side_effect` with an iterable is the standard Python answer: it converts a probabilistic fault into a deterministic script. ### Script the sequence, do not reproduce the randomness Consider a feature-flag service client that times out intermittently when the service is near its 1,200-request-per-minute peak, and application code that retries a small number of times before falling back to a cached default. The naive instinct is to make the double fail "sometimes". Never do that: a test that fails one run in fifty is worse than no test, because the team learns to ignore it. Instead state the exact history you want to exercise: ```python client.evaluate.side_effect = [ TimeoutError("peak load"), TimeoutError("peak load"), {"enabled": True}, ] ``` Each call consumes one entry. Entries that are exception classes or instances are raised; everything else is returned. So the first two calls raise and the third returns the payload — precisely the two-failures-then-recovery history, every run, in microseconds. ### Prefer instances when the handler inspects the error An exception *class* in the iterable is raised instantiated with no arguments. If the retry logic reads a message, an `errno`, a status or a `retry_after` off the exception before deciding whether the failure is retryable, supply a fully-constructed instance so that branch is actually exercised. A test that recovers from a bare `TimeoutError()` while production reads `exc.response.status` proves less than it appears to. ### Make backoff free A retry loop almost always sleeps between attempts. Leaving that sleep in place makes the test slow and, with jitter, non-deterministic. Replace the specific sleep function the module under test calls with a mock, so the delay is recorded but not taken; the recorded delays are often worth checking too, since a broken backoff calculation is a real production bug. What matters is that no wall-clock time passes. ### Get the length exactly right The iterable neither repeats nor extends. One call past the end raises `StopIteration`, and two failure modes follow from that. First, `StopIteration` is a subclass of `Exception`. A retry loop written as `except Exception: continue` — depressingly common — will swallow the exhaustion signal and keep calling, so an under-length script produces a hanging test or a bogus "all attempts failed" result rather than an obvious error. Second, since PEP 479 (Python 3.7) a `StopIteration` that escapes a generator body is converted into `RuntimeError`, so the same mistake inside generator-based code surfaces as an unrelated-looking crash. When a mock-driven test fails with a mysterious `StopIteration` or `RuntimeError`, the script being one entry short is the first thing to check. The corollary is that the script's length encodes an expectation about how many times the collaborator is called — an implicit assertion. Some teams like that; others find it too tight, because an internal refactor that adds one legitimate call breaks unrelated tests. A pragmatic middle ground is a script long enough for the intended path plus a callable tail, or a callable `side_effect` that never exhausts. ### When the callable form is better An iterable is positional: it says "the third call succeeds", not "calls for this key succeed". As soon as the code under test may reorder calls, batch them, or interleave several keys, positional scripting becomes fragile. A callable that closes over a set of already-seen keys expresses the real rule — "each key times out once, then succeeds" — and is immune to reordering. A callable is also the way to model a circuit that opens after N failures, or a stateful store that returns whatever was last written. ### Keep the test about your code The point of the exercise is the behaviour of the retry policy: how many attempts, what backoff, which exceptions are considered retryable, what happens when every attempt fails, and whether a fallback value is served. All of those are properties of your code, and a scripted `side_effect` is the cheapest way to reach them. Also script the exhaustion case — an iterable of three `TimeoutError` instances with no success at the end — because the "we gave up" branch is the one that runs during a real incident, and it is the branch most often left uncovered. These mechanics are unchanged across Python 3.10–3.14.