A contractor firm rotates its assertion-signing certificate without telling you — how does your service provider survive that?

## The failure this is about At three in the morning a contractor firm's identity provider team replaces the key it signs with. Nobody told the water authority, because from their side it was routine. Every permit sign-in from that firm now fails signature verification, and because that firm is one tenant among many, your global error rate barely moves. The engineer on call sees *SSO is broken* from one customer and nothing in the dashboards. Everything that determines whether that is a five-minute annoyance or a four-hour outage was decided months earlier, in how the connection stores keys and how it learns about changes. ## Hold a set, not a key The single most valuable design decision here is that a connection's signing material is a **set**. A verification attempt succeeds if the signature checks against any member. - An overlap becomes possible: the firm adds a new key, signs with either for a period, and retires the old one, with no coordinated cutover. - A rollback becomes possible: if the new key turns out to be wrong, the old one is still accepted. - The removal of a retired key becomes a separate, deliberate, low-risk action, taken after you have observed the new one in use. A connection that stores one certificate turns every rotation into a synchronised deploy between two organisations, one of which does not work for you. ## Consuming metadata on a schedule, or pinning a copy There are two honest positions and they trade the same risk in opposite directions. | | Scheduled refresh from the counterparty's URL | Pinned downloaded copy | |---|---|---| | Unannounced rotation | absorbed automatically | breaks that connection's sign-ins at once | | What you trust | whatever that URL serves at refresh time | what a human reviewed on the day | | Change visibility | needs an alarm on key-set change, or it is silent | every change is a reviewed action | | Operational cost | a fetch job, its failures and its monitoring | a ticket per rotation, per counterparty | Scheduled refresh scales, and it is the right default once you have more than a handful of counterparties — but it makes that URL part of your trust configuration. So: verify the host's transport certificate strictly, refuse a document whose `validUntil` has already passed, honour `cacheDuration` instead of hammering or caching forever, and **alarm when the published key set changes** so a rotation is a notification rather than a silence. Keep the last good copy on a fetch failure; a firm's metadata endpoint being down must not log its people out. Pinning is defensible for a small number of high-value counterparties where a human review per change is affordable and the extra assurance is worth the manual step. ## Your own metadata is a thing you publish and version The other half of this is the document **you** publish: an `EntityDescriptor` for your service carrying an `SPSSODescriptor`, your `entityID`, one or more `AssertionConsumerServiceURL` endpoints with their `index` and `isDefault`, a `KeyDescriptor` for any key you sign or decrypt with, and the logout endpoint you accept messages on. Counterparties consume it to configure their side. Two rules that only hurt when they are broken: 1. **A different `entityID` per environment.** Test and production must not share one. If they do, a firm configured against your test entity can have its messages accepted by production, and the mistake is invisible until it matters. 2. **Version it and publish it as an artefact.** Adding an endpoint, changing a URL or adding a key is a change every counterparty has to consume, on their schedule, not yours. Treat it as a published interface with a change log, and give yourself the same overlap you asked them for — add the new endpoint before retiring the old one, and add a new key of your own alongside the current one rather than replacing it. ## Detection and the per-connection view The operational requirement that falls out of all of this is per-connection observability: signature-failure counts, last successful metadata refresh, current accepted key fingerprints and their first-seen dates, and the age of the oldest key still in the set. That last one is the metric that stops the accepted set from silently becoming a permanent collection of everything a firm has ever used. Put a review date on each entry and remove retired keys deliberately, once the new one has been observed in real traffic. ## What you cannot do You cannot make a counterparty announce a rotation, and you should not build a process that assumes they will. You can make the unannounced case cheap — a set, a refresh, an alarm — and you can make the announced case pleasant. Everything else is a phone call you do not control.