A unit test for a subscription renewal job passes by day but fails on the nightly run. How do you diagnose it?

## The symptom is the diagnosis A test whose verdict depends on when it runs has an input that is not in the test. Nothing else can produce that pattern. So the diagnostic question is not 'why is this test unstable' but 'which unwritten input does it read', and there are only a few candidates at this level: the wall clock, an iteration order nobody promised, a locale or time-zone setting inherited from the machine, and leftover state. On a renewal job scheduled inside a six-hour nightly window, the first two dominate. ## Cause one: the clock as an ambient parameter Renewal logic is dense with date arithmetic — term end, grace period, proration, the 'due today' predicate. If either the production code or the fixture calls a global 'now', the case being tested silently changes every run. The failures cluster in recognisable places: - **midnight straddle** — the fixture computes 'today' at one instant and the code computes it milliseconds later, on the other side of a date boundary; a job starting at 23:58 and finishing after midnight reproduces this reliably and a 10:00 developer run never does; - **month and year ends** — 'one month after 31 January' has no obvious answer, and whatever the code picks, a test built on a relative date only exercises that branch on certain days; - **daylight-saving shifts** — a nominal day is 23 or 25 hours twice a year, so a test asserting a 24-hour interval is wrong twice a year and right the rest; - **time-zone offset** — the runner's zone differs from the developer machine's, and a date derived from an instant lands on a different calendar day. The confirming experiment is cheap: pin the clock to the instant printed in the failing run's log and execute the test locally. If it reproduces deterministically, you are done diagnosing — and you now have a permanent regression case. ## Cause two: the ordering assumption The second flavour is a test that asserts on a sequence the code never promised. The renewal job collects due subscriptions, processes them, and the test asserts on the resulting list positionally: ```pseudocode assert result[0].id == 4181 assert result[1].id == 4192 ``` Nothing in the production contract says which order those come back in; the arrangement happened to be stable in the developer's fixture and differs when the set is built differently or iterated by a hash-derived order. This is a false assertion, not an unlucky one: it can be red while the behaviour is entirely correct, and — worse — green while the behaviour is wrong, because it is checking a property nobody maintains. Assert on the set, or sort by an explicit key first, or, if order is genuinely part of the promise, make the production code state it and then assert on it deliberately. ## Repairing it properly The governing principle is that a test at this level may have no ambient inputs. Concretely: ```pseudocode // before: reads a global clock, so the case changes hourly function isDue(subscription): return subscription.termEnd <= Clock.now() // after: the instant is an argument, so the test states the case function isDue(subscription, at): return subscription.termEnd <= at test renewal_is_due_at_the_first_instant_after_the_term_ends: subscription = Subscription(termEnd = INSTANT_2026_03_31T23_59_59Z) assert isDue(subscription, at = INSTANT_2026_04_01T00_00_00Z) ``` Where passing an instant into every call is impractical, the equivalent move is to give the behaviour a time source it receives rather than reaches for, and hand it a fixed value in the test. Either way, the calendar becomes something the test *chooses*, which turns the previously untestable cases — month end, the shift day, the second either side of the boundary — into ordinary named cases you can write on purpose. Locale and zone deserve the same treatment: state them explicitly rather than inheriting the machine's. ## What not to do Three responses look like fixes and are not. **Widening the assertion** to a range that swallows the discrepancy destroys the oracle — the check that decides whether the behaviour is wrong — and the test now passes for the wrong reason. **Adding a delay or a retry** buys nothing at this level, because there is no asynchrony here to wait for; it converts a five-millisecond test into a slow one and leaves the ambient input in place. **Skipping the test on the nightly schedule** deletes coverage exactly where the failure was found. One more diagnostic caveat: before concluding the code is at fault, check whether the code is right and the *test's* expectation is the thing that assumed a fixed calendar. Time-dependent failures are as often over-specified assertions as they are real defects, and the distinction decides whether you ship a fix or a test change.