When a dataset is split across shards or owned by separate services so the engine cannot enforce a foreign key across the boundary, how do you decide where that invariant lives and how do you keep the data trustworthy?

## First: is the boundary real? The common failure is treating loss of referential integrity as unavoidable when it was a design choice. Two questions come first. **Did the partition key have to split this?** If orders and order lines land on different shards, the shard key is wrong. Choosing the aggregate root (customer id, tenant id, order id) as the partition key keeps the rows that constrain each other in one engine, where foreign keys, unique constraints and checks work normally. Split only across genuinely independent aggregates. **Did service ownership have to split this?** A rule that spans two services is a signal that the boundary cuts through a single consistency domain. Sometimes the right answer is moving the data, not building distributed enforcement. A principal-level answer starts here, because the cheapest cross-boundary invariant is the one that never crosses. ## When it genuinely crosses Accept that the invariant changes character. Inside one engine it was **immediate and total**: no committed state ever violates it. Across a boundary it becomes **eventual and probabilistic**: it holds most of the time, violations are possible, and your job is to bound their frequency, detect them fast, and repair them cheaply. That reframing drives the design: ### Ownership Exactly one component is authoritative. Other components hold caches or projections and must render sensibly when the referenced thing is missing — an order UI that shows 'unknown product' beats one that throws. Write the ownership down; ambiguity here is what produces two half-enforcing services and no one accountable. ### Enforcement at the write path The owner validates before writing, and does so in a way that is race-free within its own database (constraint, unique index, or a serialized write). Cross-boundary existence checks are inherently stale — the parent could be deleted a millisecond later — so the goal is 'no writes we could have prevented', not 'zero violations by construction'. ### Propagation When one side's change must reach the other, share a transaction between the state change and the notification: write the event into an outbox table in the same transaction, and publish from there. This removes the 'wrote the row, crashed before publishing' class of drift. Consumers must be idempotent, because at-least-once delivery is the realistic guarantee. ### Detection Run continuous reconciliation — periodic anti-joins or checksum comparisons between the two sides — and expose the violation count as a metric with an alert threshold. The important cultural move: a nonzero orphan count is a normal, monitored quantity, not an emergency, until it crosses the threshold. Systems that never measure this discover the drift during a migration, years of bad data later. ### Repair Decide in advance which side wins and what remediation means: backfill the missing parent, delete or quarantine the orphan, or raise it to a human queue. Money and compliance data usually get a human queue; caches and denormalized copies get automatic repair. ### Deletion policy Most cross-boundary breakage originates in deletes. Prefer soft delete or a two-phase retire (mark inactive, wait out the propagation window, then remove) so consumers get a chance to react. A hard delete on the parent side with no notification is the classic orphan factory. ## The tradeoff to state out loud What you bought: independent scaling, independent deploys, blast-radius isolation, no cross-shard write coordination. What you paid: a class of correctness that used to be free is now an operational program — reconciliation jobs, metrics, repair runbooks, and a permanent design tax on every consumer that must handle missing references. Distributed transaction protocols (two-phase commit) can restore immediacy but couple availability across the boundary, which usually defeats the reason you split. An interview answer that names this trade honestly — and refuses to pretend an eventual invariant is the same as a declared one — is the differentiator. The weak answer says 'we just check it in the service'; the strong one says which invariants stayed engine-enforced by construction, which became eventual, who owns them, and how a violation is noticed within minutes rather than years.