What is a synthetic check, and how does it differ from a test in the build pipeline?
answer
- Runs after release, not before
- Triggered by a clock, not a commit
- Same assertions, live target
- Only a few critical journeys
- Dedicated identity, ring-fenced records
basics
~20 sA synthetic check is a scripted transaction run against the live system on a fixed interval by a scheduler outside it, alerting when it fails. A pipeline test runs once per change, against a build.
solid answer
~50 sA synthetic check is a small scripted journey - log in, request a payslip preview, assert the response - that a scheduler executes against the running production system every few minutes, from outside it. The assertions look like ordinary test assertions; what differs is the trigger (a clock, not a commit), the target (the deployed system with its real configuration, certificates, dependencies and data) and the consequence (an alert and a runbook, not a red build). That makes it the only test that can catch failures which never existed at build time: an expired certificate, a mis-set configuration value, a dependency that started returning errors overnight. Coverage is deliberately thin - a handful of journeys that matter most - because every check costs real traffic, real data and on-call attention. It must run against dedicated accounts and ring-fenced records, tagged so downstream reporting can exclude it.
code
pseudocode · 9 linesevery 5 minutes:
session = login(SYNTHETIC_EMPLOYEE_ID, secret("synthetic-check-password"))
payslip = session.get("/payroll/preview?period=current", header("X-Traffic", "synthetic"))
assert payslip.status == 200
assert payslip.gross == payslip.net + sum(payslip.deductions)
assert payslip.elapsed_ms < 2000
emit_result(check="payslip-preview", region=here(), tag="synthetic")go deeper
Be ready to define a synthetic check in one sentence and name the three differences from a pipeline test: it is triggered by a clock, it targets the live deployed system, and its failure pages someone instead of blocking a merge.
Explain what a check can catch that no pre-release test can - expired certificates, wrong deployed configuration, a degraded third-party dependency - and why coverage is kept to a few journeys rather than mirroring the regression pack.
Show that you have operated them: dedicated identities and ring-fenced data, tagging so synthetic traffic never pollutes telemetry, alerting on consecutive failures rather than single blips, and noticing when a check has been quietly weakened into a permanent green.
Own the policy question - which journeys earn a check across the whole estate, what a check failure obliges the on-call rota to do, and how you stop check count and alert noise growing until nobody believes the signal.
### The idea A synthetic check is a test whose subject is the running system rather than a build artefact. Somebody writes a short script - authenticate as a known identity, call an endpoint or drive a screen, assert on what comes back - and a scheduler runs that script against production on a fixed cadence, typically every one to five minutes. The traffic is *synthetic* because no real user generated it; the term *probe* and the phrase *active monitoring* mean the same thing. It contrasts with passive monitoring, which only observes traffic that real users happen to send. ### Why the trigger and the target matter A pipeline test and a synthetic check can contain literally the same assertion. The interesting differences are structural. **Trigger.** A pipeline test runs once per change. Between two changes it says nothing. A synthetic check runs on a clock, so it keeps asserting during the long periods when nobody is deploying - which is exactly when certificates expire, disks fill, credentials rotate, upstream providers degrade and scheduled jobs misfire. **Target.** A pipeline test runs against a build in an environment you constructed for testing, usually with fabricated data and stubbed dependencies. A synthetic check runs against the deployed system: the real configuration values, the real network path, the real third-party integrations, the real data. Whole categories of defect exist only there. A payroll engine can pass every pipeline test and still fail in production because the deployed instance points at last quarter's tax-table service. **Consequence.** A failing pipeline test blocks a merge. A failing synthetic check pages a human. That difference disciplines the design: the check must be reliable enough that its failure is believable, and it must fail in a way that tells the responder what to do. ### Coverage discipline The instinct after writing the first check is to script everything. Resist it. Each check consumes production capacity, writes or reads production data, and - most expensively - buys a claim on the attention of whoever is on call. A useful rule is to cover the journeys whose failure you would want to hear about at three in the morning, and nothing else. For a payroll engine that is perhaps four: sign-in, payslip preview, submitting a pay run, and the outbound payment handoff. Everything else is better served by the regression suite before release and by telemetry after it. Thin coverage has a consequence you must state out loud: green checks do not mean users are fine. They mean four journeys, exercised with one data shape from one location, are fine. When an incident is reported that the checks missed, the correct response is to treat the incident as a specification for a new check - not to conclude the checks are worthless. ### Where the check runs from Running the check from inside the same deployment as the application proves the application logic and little else. Running it from outside - a separate network, ideally more than one geography - additionally proves name resolution, certificate validity, edge routing, load-balancing and authentication at the boundary. The cost is that the check now shares a failure domain with the public network, so it can fail for reasons that have nothing to do with your system. Many teams run both: an internal check that is the trustworthy signal, and an external check that reflects what a user would experience. ### Accounts and data This is the part junior candidates skip and interviewers probe. A check that runs in production touches production data, so its blast radius has to be designed: - **Dedicated identities.** A synthetic employee record the team owns, never a real person's account, never a shared administrator credential. - **Ring-fenced records.** Data the check writes must be invisible to anything that consumes real data - reports, month-end totals, outbound filings, the payment batch. - **Tagging.** Every synthetic request carries a marker so telemetry, billing and analytics can exclude it. Untagged synthetic traffic quietly poisons the baselines you later want to use as an oracle. - **Credentials.** The check's secrets live in the same managed store as everything else and rotate on the same schedule; they do not live in the check's source. - **Reversibility.** Prefer read-only paths. Where a check must write, it should write something self-expiring, or clean up after itself, and it must never trigger an irreversible outbound side effect such as an actual disbursement. ### Operating it A check is production code and deserves the same treatment: version control, review, its own deployment, and a health signal of its own so a check that has silently stopped running is distinguishable from a check that is passing. Alert on a pattern - two or three consecutive failures, or failures from more than one location - rather than on a single blip, or the noise will train people to ignore it. And watch for the slow decay where someone adds a retry, then relaxes an assertion, until the check cannot fail: at that point it is a green light with no meaning behind it, and it is worse than having no check at all.
- Which accounts and data should a synthetic check be allowed to touch in production?Dedicated synthetic identities the team owns, never a real person's record and never a shared administrator credential. The data it writes must be ring-fenced so no report, total or outbound batch consumes it, and every request tagged so telemetry and billing can exclude it. Secrets come from the managed store on the normal rotation. Prefer read-only paths; where the check must write, the write should be reversible or self-expiring, and it must never trigger an irreversible outbound side effect.
- Where should a synthetic check run from, and how does that change what it proves?From inside the deployment it proves the application path and little else. From outside - a separate network, ideally several geographies - it also proves name resolution, certificate validity, edge routing and boundary authentication, which is closer to what a user experiences. The cost is shared failure with the public network, so an external check can fail for reasons unrelated to your system. Many teams run both and treat the internal one as the trustworthy paging signal.
- The synthetic checks are all green but users are reporting failures - what do you conclude?That the checks cover a narrow slice: a few journeys, one data shape, one identity, one or two locations. Real users bring other roles, locales, devices and record shapes. Treat the incident as a specification for a new check rather than as evidence the checks are useless. Also inspect the checks themselves - if someone has added retries or loosened an assertion over time, the green result may no longer mean anything.
A pipeline test is the inspection a vehicle passes before it leaves the factory. A synthetic check is the driver who takes it once round the block every morning, on real roads, before anyone else gets in.
saying these in an interview costs you the question
- Claims synthetic checks replace the regression suite
- Runs the check against a real customer's account
- Adds retries until the check stops failing
- Scripts fifty journeys instead of the few that matter
- Treats every check failure as a false alarm
- Assumes green checks mean no user is affected