skip to content

Why must the clock be an injected input for tests that post a SAML assertion to your AssertionConsumerService?

level: seniorimportance: should knowfreq 27%

answer

  1. every check compares against now
  2. fixtures expire, builds go red
  3. both sides of every bound
  4. the tolerance is only pinned by two tests
  5. derive bounds from an injected position

basics

~20 s

Because every bound a federated sign-in enforces is clock-relative. With the system clock, a fixture's validity window expires overnight and the just-inside and just-outside cases of each bound and of the skew tolerance are impossible to express at all.

solid answer

~40 s

A federated login is decided almost entirely by comparisons against now: the `Conditions` `NotOnOrAfter`, the `SubjectConfirmationData` `NotOnOrAfter`, any `SessionNotOnOrAfter` you carry into your local session, the connection metadata's `validUntil`, and whatever clock skew you chose to tolerate. If the verifier reads the system clock, two things follow. Recorded fixtures rot: a message captured on Tuesday fails on Wednesday for a reason that has nothing to do with the code. And the interesting cases vanish: you can test the happy middle of a window but not one second inside the bound, one second outside it, or either side of the tolerance you allow. Make the clock a dependency the suite sets, generate fixtures relative to that position, and assert both sides of every boundary.

code

pseudocode · 20 lines
pseudocode
# one fixture, four clock positions, four stated outcomes
T = clock.now() + 5 * MINUTE
msg = standIn.sign(subject = "u-417",
                   conditionsNotOnOrAfter = T,
                   subjectConfirmationNotOnOrAfter = T)

clock.set(T - 1 * SECOND)
assert acs.post(msg).issuedSession()                 # inside the window

clock.set(T)
assert acs.post(msg).refused("expired")              # NotOnOrAfter excludes T itself

clock.set(T + ALLOWED_SKEW - 1 * SECOND)
assert acs.post(msg).issuedSession()                 # inside the tolerance you chose

clock.set(T + ALLOWED_SKEW + 1 * SECOND)
assert acs.post(msg).refused("expired")              # outside it

# note: the replay record is not exercised here - each post above
# is preceded by a fresh assertion id from the builder.

go deeper

for a junior

Recall that a federated sign-in message is only good for a short time, so a test that uses the real clock will start failing on its own. The test needs to control what time the server thinks it is.

for a middle

Name the separate bounds and who sets each, and explain why the fixture builder should derive them from the injected clock rather than carrying fixed timestamps that must later be edited.

for a senior

Demonstrate boundary discipline: accepted one second inside, refused at the bound, accepted just inside the tolerance, refused just outside it — and explain why the last pair is what stops the tolerance drifting.

for a principal

Treat clock control as the precondition for the fixture corpus surviving years rather than months, and make the standing call on which time-dependent behaviour is asserted in tests versus monitored in production.

## Time is an input to this decision, not an environment detail The rota tool accepts a sign-in from a venue operator's identity provider on the strength of a handful of checks, and most of them are comparisons against the current time. There is no way to test a comparison you cannot position yourself on both sides of. So the clock the verifier reads must be a dependency the test supplies, in the same way the registered signing certificate is, and the fixtures must be minted **relative to** that position rather than carrying absolute timestamps from whenever somebody captured them. ## Which windows, set by whom, bounding what This is the ambiguity that sinks a careless suite: there are several time bounds in play, each set by a different party and bounding a different thing. | Bound | Set by | Bounds | |---|---|---| | `Conditions` `NotOnOrAfter` | the identity provider | how long the assertion as a whole may be considered valid | | `SubjectConfirmationData` `NotOnOrAfter` | the identity provider | how long this delivery of it may be presented to your endpoint | | `SessionNotOnOrAfter` | the identity provider | how long the provider vouches for the session it created | | metadata `validUntil` / `cacheDuration` | the counterparty publishing metadata | how long their published document may be relied on or cached | | your local session expiry | you | how long your own session record lives | | the skew you tolerate | you | how much disagreement between two machines you absorb | A test that says "the assertion expired" without saying which of these bounds it moved past is testing something it cannot name. Worse, the replay record you keep to refuse a second post is TTL-ed off exactly one of these, so getting the wrong one wrong produces either an unbounded store or a window in which a replay is accepted. ## The two failures a system clock produces **Fixtures rot.** Any fixture that carries an absolute time-bounded element stops being accepted the moment that bound goes into the past. A suite built on recorded messages then fails overnight, in a build nobody changed, and the standard reaction is the worst possible one: widen the tolerance until it passes. Now the tolerance is a number nobody chose, and it is in production. **The boundary cases are inexpressible.** With a controllable clock the suite can state, precisely: 1. One second before the bound — accepted. 2. At the bound — refused, because `NotOnOrAfter` excludes the instant itself. 3. Just inside the skew you chose to allow beyond the bound — accepted. 4. Just outside it — refused. Steps 3 and 4 are the pair that actually pins the tolerance. Testing only the happy middle leaves the tolerance free to be changed by anyone, at any time, with a green build. ## Generated relative to now, not recorded The practical consequence is that the fixture builder takes the clock as a parameter and derives every bound from it. A test that wants an expired message does not edit a timestamp; it asks the builder for a message whose `Conditions` bound is a minute behind the clock position, or leaves the fixture alone and advances the clock. Both read clearly; the second scales better, because one fixture then serves the accepted case, the expired case and the boundary pair. There is a second-order benefit. Because the stand-in issuer signs at mint time and the bounds are inside the signed material, a builder that derives bounds from an injected clock keeps every fixture consistently signed at whatever position the test chose. Editing a timestamp inside an already-signed document does not — it produces a signature failure, and the test then passes for entirely the wrong reason. ## What this does not license Freezing the clock makes the boundaries testable; it does not make them correct. Deciding how much skew to absorb, how long the replay record lives and what your own session expiry should be are design decisions with their own homes. The testing question is narrower and answerable: whatever you chose, is it asserted on both sides, and does the suite still pass in six months without anyone touching a number?

  • A federated sign-in test starts failing overnight with nothing changed. What is the first thing to check?
    Whether the fixture carries absolute time bounds and the verifier is reading the system clock. That combination fails on a calendar rather than on a defect. Fix it by injecting the clock and deriving the fixture's bounds from the test's chosen position — not by widening the skew tolerance, which silently changes production behaviour to make a build green.
  • Why is editing a timestamp inside a recorded, signed assertion a bad way to build an expired fixture?
    Because the bounds sit inside the signed material, so the edit breaks the signature. The endpoint then refuses the message for a signature failure, the test passes, and it is asserting nothing about expiry. Mint the fixture from the stand-in issuer at the clock position you want, or leave the fixture alone and move the clock.

saying these in an interview costs you the question

  • Widens the skew tolerance to make a stale fixture pass again.
  • Tests only the middle of a validity window and calls it covered.
  • Says "the assertion expired" without naming which bound was crossed.
  • Edits a timestamp inside an already-signed fixture to make it expired.
  • Treats a nightly-failing federated test as environmental flakiness.
  • Assumes the local session expiry and the provider's bound are the same clock.