Why does Census need a writable schema in your warehouse, and what does that cost?

## Why any diffing sync needs state A reverse-ETL sync's job is to make a destination agree with a model. Doing that naively means sending every row every run, which destinations will not tolerate: their APIs are rate-limited, often metered, and frequently slower than the warehouse by orders of magnitude. So the sync must answer "what changed since last time?", and answering it requires remembering last time. That memory is the bookkeeping state. Census keeps it in your warehouse, in a schema you grant it write access to, rather than copying your data out to vendor-side storage. The privacy argument is the appealing one — your customer records stay inside your own account and only the rows actually being activated leave it. The operational argument is the one interviews care about: this makes sync state a table in a database your team administers, with everything that implies. ## What each run actually does Roughly: run the model's query, materialize or scan the result, compare it against the recorded state for this sync, compute the set of inserts and updates (and, under a mirror-style behavior, removals), send those to the destination's API, then record the new state including which records the destination rejected. The comparison is warehouse work. On a wide model with millions of rows it is not free, and it happens on every scheduled run whether or not anything changed. ## The cost profile Think of it in three parts. **Warehouse compute** — a scan and a comparison per sync, so cadence multiplies cost. A sync every fifteen minutes over a hundred-million-row model is a standing query workload you have to budget for, and the standard mitigation is to align the schedule to how often the model can actually change rather than to how fresh someone wishes it were. **Warehouse storage** — modest, but real, and it grows with the number and width of syncs. **Destination API consumption** — governed by how many rows the diff produced, which is normally small and occasionally enormous. A useful design instinct: make the model narrow. Only map the fields you truly want the warehouse to own, and only select those columns. Every extra column is another value that can change and trigger a row into the diff, so wide models produce chattier syncs. ## The failure mode: a full resend The operational story worth telling is the morning where a normally-quiet sync pushes the whole model. Causes cluster into a few shapes. The bookkeeping state was lost — a warehouse cleanup script dropped an unfamiliar schema, an environment was recloned, a sync was recreated from scratch. A full resync was triggered on purpose, sometimes to repair a suspected drift, without anyone checking the destination's remaining quota. Or the model's values genuinely all changed at once, because a transformation was refactored, a rounding or timezone conversion shifted every value, or a new column was mapped into the sync. The effect is the same either way: every row looks new, the destination throttles, the sync stretches for hours or fails partway, and other integrations sharing that org's API allowance start failing too. The blast radius extends past your pipeline, which is what makes it a senior question. ## Operating around it Treat the bookkeeping schema as pipeline infrastructure: exclude it from warehouse cleanup automation, from clone-and-scrub scripts and from cost-driven table pruning, and make sure the people who administer the warehouse know what it is. Understand your destination's rate limits and remaining allowance before triggering a deliberate full resync, and prefer to run one out of business hours. Refactor transformation logic knowing that a change touching every row is also a change pushing every row, and stage such changes rather than shipping several at once. Watch the per-run record count, not just success and failure — a run that sent a hundred times its usual volume is an incident even when it eventually completes. ## How to answer Start with the mechanism (diffing needs prior state, and the state lives in your warehouse), give the privacy rationale briefly, then spend most of the answer on consequences: warehouse compute scales with cadence and model width; the state is load-bearing infrastructure; losing or resetting it produces a full resend that collides with the destination's rate limit and hurts other integrations sharing it. That progression — mechanism, cost, failure mode, mitigation — is what distinguishes someone who has operated the tool from someone who has read the marketing page.