Why is a suite that calls a vendor-hosted mock service for a seed-bank inventory API not hermetic?

## What hermetic actually promises A **hermetic** test is a test whose outcome is determined entirely by the code and data in the checkout, plus whatever the test itself starts. Run it on a laptop with the network unplugged, run it on a fresh machine, run it next year: same input, same verdict. That property is what makes a failing test *information*. If the only thing that can change the result is your change, then red means your change is wrong. A vendor-hosted mock service is a stub server the vendor operates, with your mapping set held in an account on their platform. Pointing a suite at it is convenient, and it is not hermetic, for reasons worth separating — because they fail differently and they are fixed differently. ## The dependency list grows Running a suite against a hosted mock quietly adds to the set of things that must be healthy for a green build: - **The network path** from wherever the tests run — a laptop on hotel wifi, a runner inside a locked-down network — to the vendor's address. - **The vendor's service itself**, whose incidents you learn about at the same moment your build does. - **The account's standing**: an expired credential, a lapsed subscription or an administrator's tidy-up all present as failing tests. - **A shared address**, which brings shared capacity and shared throttling behaviour that your suite does not control. None of these are exotic. The point is that each can produce a red build with no defect behind it, and none of them are repaired by editing code. ## Shared state defeats isolation The deeper problem is not availability but sharing. The mapping set in the account is a shared mutable thing, and everything pointing at it sees the same version of it: - A colleague adjusting a reply so their branch passes changes what your branch sees, immediately, with no merge and no notification. - Parallel workers that each need a different reply for the same request cannot both have it. Whichever wrote last wins, and the loser fails for a reason its own code cannot explain. - State outlives the run. A test can pass because an earlier run left the account configured helpfully, which is the most expensive kind of green: it disappears the day somebody tidies up, and the failure it was hiding surfaces somewhere unrelated. A stand-in you start yourself avoids all of this for free, because the instance dies with the test. Sharing is exactly the property you bought when you chose a hosted service, and it is exactly the property that costs you isolation. ## What red means afterwards The consequence teams actually feel is semantic. Once a red build can mean *your change is broken* or *the vendor is having a bad afternoon*, the team must triage before it can read, and triage is expensive at the worst moments: during a release, late at night, when the person who understands the account is asleep. The predictable outcome is the re-run reflex. People learn that hitting retry sometimes works, so they retry before they read the logs, and that habit persists into the reds that are real. A suite that is right most of the time but ambiguous some of the time is often less useful than a smaller suite that is unambiguous. ## Routing calls deliberately The answer is rarely *never use it* and never *route everything there*. Decide per call: - Anything exercised by fast, frequently-run tests should stand in locally, where the definitions are yours and the instance is disposable. - A hosted mock earns its place where the value is precisely that it is shared and always up: a stand-in a mobile client, a partner team or a demo environment can all reach without running anything. - Release gates should depend on the fewest externally owned parts you can justify, because that is the moment when an unexplained red costs the most. ## Making the dependency honest If you keep the dependency, make it visible rather than pretending it away: - Put those tests in their own suite on their own schedule, so a vendor incident cannot block a merge queue that has nothing to do with it. - Fail fast and say why. A readiness check against the address at start-up turns a confusing cascade of assertion failures into a clear message naming the vendor as the cause. - Keep an exported copy of the mapping set in your repository, so *the replies changed* is answerable from a diff rather than from memory. - Give the suite an owner, because an externally owned dependency with no internal owner is how a red build becomes background noise. Hermetic is not a moral standard, and plenty of valuable tests are not hermetic. But it is worth naming out loud what you traded away, because the price is paid in the currency of trust in the suite, and that currency is hard to earn back.