What does a payment provider's own sandbox prove that a stub you write yourself cannot?

## What each arrangement actually is A **stub you write** is a process you start, populate and stop. You author its replies, so it agrees with you by construction: it returns what you told it to return, to the requests you anticipated, in the order you arranged. That is a feature — it is what makes a suite repeatable — and it is also the ceiling on what such a suite can discover. A **provider sandbox** is the third party's own system running in test mode. The provider wrote the code, the validation and the state transitions; you are a guest holding credentials they issued. Nothing in it agrees with you by construction, and that disagreement is the entire product. ## What the sandbox proves - **Your request is acceptable to code you did not write.** Required combinations, mutually exclusive options, encodings, signing and the exact shape of an authenticated call are all enforced by the provider. A stub you authored cannot enforce a rule you never knew existed. - **Their refusals are real refusals.** What comes back is what a real caller gets, phrased in their vocabulary, carrying whatever detail they actually include — often far less than the helpfully complete error body a developer writes into a stub. - **Their sequencing is real sequencing.** Where an operation must follow another, cannot be repeated, or settles asynchronously, the sandbox enforces that ordering. A stub enforces only the ordering you remembered to encode. - **Your credential handling works end to end.** The call is authenticated against a real issuer, so a broken signature, an expired-token path or a misconfigured client surfaces in a test rather than in production. - **Timing is approximately real.** Callbacks arrive when they arrive, over a real network, and the sandbox will not deliver them at your convenience. ## What it cannot prove - **It is not production.** It is a separate deployment with its own data, its own configuration, its own risk and eligibility settings, and often a different build of the provider's service. Passing there constrains release day; it does not determine it. - **A green run says nothing about tomorrow.** Nothing about passing this afternoon restrains what the provider ships tonight, and no commit in your repository will explain the change. - **It cannot reach what they did not expose.** Unusual outcomes are reachable only where the provider publishes a designated trigger for them. A partial outage, a malformed reply, or a connection dropped mid-body is not on the menu; you produce those yourself with a stand-in. - **It cannot exercise your failure handling under repetition.** Hammering an operation until every branch of your retry logic has run is precisely the usage the account's throttling exists to prevent. ## The comparison, laid out | Question | A stub you control | The provider's sandbox | |---|---|---| | Who wrote the answers? | You did | The provider did | | Can it refuse something you never anticipated? | No | Yes | | Is a run repeatable and self-contained? | Yes | No | | Can you reach an arbitrary edge case? | Yes, by authoring it | Only where a trigger is published | | Does a green result mean production is safe? | No | Still no | | Who can change its behaviour without telling you? | Nobody | The provider | ## How teams combine them The mature shape is a split rather than a choice: - The great majority of tests run against a stand-in you control: repeatable, fast, parallel, and free to model faults the provider will never produce for you. - A small, deliberately chosen set runs against the sandbox, usually on a schedule rather than on every commit, and answers a narrow question — does the provider still accept what we send, and do we still understand what they send back? - Failures in that scheduled set are triaged by a person, because a build cannot distinguish your defect from their change or their outage. ## The mistake worth naming The common failure is the team that adopts the sandbox and then routes everything to it, on the reasoning that more realism must always be better. What they get instead is a suite whose runtime is a network round trip per assertion, whose failures are ambiguous, whose parallelism is capped by somebody else's throttling, and which still cannot exercise the failure modes that actually take a service down. Realism at the edge is valuable precisely because it is rationed. Spread thinly across every test, it buys very little and costs the team its fast feedback loop — and, worse, it teaches everyone to distrust a red build, which is the habit that lets a genuine integration break sit unnoticed for a week.