Your hosted scheduled collection run is failing while your own runs of the same collection pass. How do you triage that?

## Why the naive conclusion is wrong The tempting inference is: the hosted run is red, my run is green, therefore the hosted run is unreliable. That inference is unsafe in both directions. The hosted run may be seeing something real that your run structurally cannot see, or it may be failing for a reason that has nothing to do with your service at all. The reason you cannot tell yet is that **the two runs differ in more places than the collection**. So triage here is not about the requests. It is about enumerating the substitutions the arrangement made and eliminating them in order. ## The five substitutions, and what each one breaks | Substitution | What differs | The failure it produces | |---|---|---| | **The copy** | the hosted account holds its own collection | asserts against requests or expectations you have since changed | | **The environment** | values stored beside the hosted copy | points at a different target, or carries a stale value | | **The identity** | credentials kept in that account | authentication failures after a rotation you performed elsewhere | | **The origin** | the request enters from the public side | name resolution, edge rules, filtering and throttling your in-network run never meets | | **The driver** | their clock and their machines | a run that started late, was retried, or did not start | A candidate who can produce that table has answered the question, because triage is then simply walking it. ## The order to walk it in 1. **Read the failure itself first.** Which request, which assertion, and is it the same request failing every time or a different one? A single request failing consistently points at that path; scattered failures point at identity, origin or the driver. 2. **Take the hosted copy and run it yourself.** Not your repository copy — the copy the schedule actually holds. If it fails for you too, this is copy drift or a real defect, and the origin is innocent. 3. **Check the environment it used.** Confirm the target it names is the target you think, and that the values it carries are current. A hosted check quietly pointed at a different environment produces failures that make no sense against the one you are looking at. 4. **Check the identity.** If the failure is an authentication or authorisation failure, the credential stored in that account is the first suspect, particularly if anything was rotated recently on your side. This is the single most common cause of a schedule that goes red without any deploy. 5. **Reproduce from outside your network.** If the hosted copy passes from inside and fails from outside, the difference is the path in. That is precisely the vantage point you bought by putting the check there, and the finding is a real one about how your system answers strangers. 6. **Only now suspect the service.** And when you do, remember that a run entering from outside may reach a different instance, a different edge, or a different set of rules than a run started inside. ## What to do with the answer - **If it is the copy:** republish the hosted copy from the authoritative repository copy, and note how long the drift had gone unnoticed — that number tells you how much to trust every other hosted check. - **If it is the identity:** the rotation process is incomplete, because it does not include a store you do not hold. That is a process defect, not a one-off fix. - **If it is the origin:** you have found something your own pipeline could not have found. Treat it as a genuine defect in the path, and be precise about which hop it lives in. - **If it is the driver:** you have learned that your feedback loop has a dependency on somebody else's availability, and that its silence needs its own detection. - **If it is the service:** the hosted check did its job, and the only remaining question is why the same failure was invisible from inside. ## Boundaries worth naming out loud Saying where a question stops is itself a senior signal. Two neighbouring subjects are not this one: - **Deciding whether an intermittent failure is a real defect or an unreliable case** is a testing discipline with its own criteria, and "it is flaky" is not a triage step — it is a conclusion you may only reach after the five substitutions have been eliminated. - **Whether this check is a good indicator of service health**, and what it misses compared with real traffic, is a service-level measurement question owned elsewhere. What belongs here is the discipline of remembering that the hosted run and your run share only the requests, and everything around them was swapped.