When you stand in for a venue operator's identity provider in tests, why must the stand-in really sign its assertions?
answer
- the rejections are the point
- you cannot forge without a key
- one builder, many mutations
- faithful on what the verifier inspects
- throwaway keypair, registered as the certificate
basics
~20 sBecause the checks worth testing are rejections, and a stand-in that does not sign cannot produce a message that is signed by the wrong key. Holding a test keypair lets the suite mint both the accepted message and every rejected variant of it.
solid answer
~50 sA double that returns a canned, unsigned document only lets you assert the happy path, and only if you first weaken the verifier to accept it — which deletes the check you meant to test. A stand-in that owns a keypair the suite generates, registered against a test connection as that venue's signing certificate, lets you mint the valid assertion **and** its mutations from one builder: signed by a different key, carrying a different audience, carrying a different issuer, unsigned, expired, or correlating to no request you made. Each mutation is one named test asserting one specific rejection. The stand-in must be faithful about the things the verifier inspects — the signature, the `AudienceRestriction`, the validity bounds, the `NameID`, `InResponseTo` — and may fake everything else, including the human, the login screen and the directory behind it.
code
pseudocode · 25 lines# one stand-in issuer, one builder, many mutations
testKeys = generateKeyPair() # throwaway, per suite run
registerConnection(venue = "north-pier", signingCert = testKeys.public)
def assertionFor(subject, **over):
a = buildAssertion(
nameId = subject,
issuer = over.get("issuer", "https://idp.north-pier.test"),
audience = over.get("audience", ROTA_ENTITY_ID),
notOnOrAfter = over.get("expiry", clock.now() + 5 * MINUTE),
inResponseTo = over.get("corr", lastRequestIdIssued()))
return sign(a, over.get("key", testKeys.private))
# positive
assert acs.post(assertionFor("u-417")).issuedSession()
# negatives: one mutation each, one named refusal each
assert acs.post(assertionFor("u-417", key = generateKeyPair().private)).refused("signature")
assert acs.post(assertionFor("u-417", audience = OTHER_ENTITY_ID)).refused("audience")
assert acs.post(assertionFor("u-417", issuer = "https://idp.pier-road.test")).refused("issuer")
assert acs.post(stripSignature(assertionFor("u-417"))).refused("unsigned")
assert acs.post(assertionFor("u-417", corr = "never-issued")).refused("correlation")
for r in [acs.post(m := assertionFor("u-417")), acs.post(m)]:
pass
assert r.refused("replay") # the second post of the identical messagego deeper
Recall that the service provider trusts a federated sign-in because of a signature made by the identity provider. A test double that does not sign cannot produce a badly signed message, so the check on that signature goes untested.
Explain the keypair arrangement: the suite generates one, registers the public half as that connection's signing certificate, and mints fixtures from one builder with overrides for audience, issuer, validity and key.
Show that you assert specific refusals rather than generic failure, including the validly-signed-but-wrong-scope shape only a key-holding stand-in can produce, and that no local session survives a rejection.
Weigh what fidelity costs. Decide how far the stand-in should model a real counterparty's quirks before that modelling becomes its own untested product, and where conformance to the specification is asserted instead.
## The design decision, not the tool choice The useful question is not which stub server you run. It is what the stand-in has to be **faithful** about. A federated sign-in is a trust decision, and the whole decision rests on a cryptographic signature made by a party you do not control. If your stand-in does not sign, the only way your endpoint accepts its output is if you turn signature verification off in the test profile — and the moment you do that, the test exercises a configuration that will never run in production and cannot fail for the reason production would. So the stand-in generates a keypair, the suite registers the public half against a test connection as that venue operator's assertion-signing certificate, and every fixture is signed for real. The keypair is throwaway, generated per suite run, and is emphatically not the service provider's own signing or decryption key and not the TLS certificate on the endpoint — those are different keys with different owners and different rotation stories. ## Faithful about what, fake about what | The stand-in must be faithful about | It may fake entirely | |---|---| | Producing a real signature over the real bytes | Any login screen or password prompt | | The element the signature covers | Whether a human was present | | `AudienceRestriction` naming your `entityID` | The venue's directory or HR system | | `Conditions` and `SubjectConfirmationData` `NotOnOrAfter` | Multi-factor, consent, or risk checks | | A `NameID` and the attributes your mapping reads | Session state at the identity provider | | `InResponseTo` echoing a request you issued | Any endpoint your code never calls | The test is not about the venue operator's authentication quality; it is about what your service provider does with what arrives. ## The negative cases are the deliverable Once the stand-in signs, a single fixture builder with overrides yields the suite that actually matters: - **Signed by the wrong key** — a second, unregistered keypair. Must be refused as a signature failure, not as a parse error. - **Wrong audience** — `AudienceRestriction` naming a different service provider. This is the check that catches an assertion minted for somebody else's tenant. - **Wrong issuer** — a message whose `iss` is a venue connection other than the one it was posted for. - **Unsigned** — the document with the signature stripped. Must be refused, never silently accepted as "nothing to verify". - **Expired** — a `NotOnOrAfter` already in the past at the test's clock position. - **Uncorrelated** — an `InResponseTo` naming a request identifier you never issued. - **Replayed** — the byte-identical accepted message posted a second time. Each asserts a **specific** outcome. "It threw" is not an assertion: a typo in the fixture builder also throws, and a suite that only checks for failure will happily stay green after the check under test has been deleted. Assert the rejection reason and the response the endpoint gives, and assert that no local session was issued. ## The trap in "it rejected it" The subtlest defect this suite catches is a message that is validly signed but signed over the wrong part of the document — a structure that verifies cleanly while the data your code reads comes from somewhere the signature never covered. Your stand-in can produce that shape deliberately because it holds the private key. A recorded fixture from a real counterparty never will, because a real counterparty never sends one. ## Where this stops being yours Two boundaries are worth stating out loud. First, the rules a signature must satisfy — what must be covered, how encryption composes with it — are the specification's, and your test asserts conformance to them rather than restating them. Second, how you start a stub server, tell it which requests to answer, and whether it runs inside or outside the test process is generic tooling that has nothing to do with identity; the identity-specific part is what the stand-in signs, what it claims, and which mutation of it you expect to be refused. ## What good sounds like "We generate a keypair per suite run and register it as that connection's signing certificate, so the stand-in really signs. Then the value is in the mutations: wrong key, wrong audience, wrong issuer, unsigned, expired, uncorrelated, replayed — each one a named test asserting a specific refusal and no session. A double that cannot sign forces you to disable the verifier, and then the test proves nothing about the code that runs in production."
- Which keys are in play here, and which one does the suite generate?Four: the identity provider's assertion-signing key, the service provider's own signing and decryption key, the TLS key on the endpoint, and the stand-in's throwaway test keypair. The suite generates only the last and registers its public half as the test connection's signing certificate. The first belongs to a customer you cannot phone; conflating it with the others is how a test ends up asserting the wrong rotation.
- Why is asserting "the request failed" not good enough for a negative case?Because a malformed fixture, a builder bug or an unrelated exception all fail too, so the assertion stays green after the check it was written for has been deleted. Assert the specific refusal — signature, audience, issuer, expiry, correlation, replay — and assert that no local session was issued and no account was created.
- Should the stand-in model the quirks of one particular venue operator's output?Only where a quirk is a divergence you have decided to absorb, and then it gets a named test explaining why. Modelling quirks generally turns the stand-in into a second product nobody tests. Keep it conformant by default and let each deliberate divergence be visible as its own fixture.
saying these in an interview costs you the question
- Uses a double that returns an unsigned document and disables verification in tests.
- Says the stand-in must reproduce the identity provider's login screen.
- Reuses the service provider's own signing key for the stand-in fixtures.
- Asserts only that a bad message failed, never which check refused it.
- Believes a real counterparty's recorded message can stand in for a wrong-key case.
- Treats the stand-in as the thing under test rather than the counterparty.