When should data movement be split out of the schema change set rather than shipped inside it, and what does that split cost?

## Why the question exists at all A schema change is usually near-instantaneous; the data movement attached to it is not. When both live in the same ordered change set, the release waits for the slow half. That has three consequences a lead has to weigh: the deploy's duration becomes the data's duration, an abort part-way through leaves an ambiguous state, and every instance waiting to start is waiting on a data run rather than on a structural change. So there is a genuine architectural choice: **is the data movement part of the release, or a separate piece of work the release merely permits?** ## When keeping it inside is right It is not automatically wrong to ship data movement inside the change set — for most changes it is the simplest correct answer. Keep it inside when: - the affected row count is small enough that the run is bounded and predictable; - the new code **cannot** function against un-migrated rows, so the release must not begin serving until the data is complete; - the movement and the structure are one logical change and splitting them creates a window nobody would otherwise have to reason about; - the data is reference or lookup rows, which are tiny and are part of the schema's meaning. The advantage of staying inside is that the history remains a complete description: replay it from empty and you get the same database, structure and data alike. ## When to split it out Split it when both halves of this are true: 1. **The run is long enough to matter** — long enough that holding the release open for it is unacceptable, or that it must be startable, stoppable and restartable on its own schedule. 2. **The release is correct without it.** The new code must handle a row whose new value is absent: derive it at read time, fall back to the old column, or treat absence as a defined state. If it cannot, splitting does not remove the dependency — it converts a slow deploy into a partially broken one. The second condition is the one teams skip. The honest test is to ask what a user sees for a row the run has not reached yet, and to be able to answer without saying "the run will be finished by then". ## What the split costs | Cost | Why it bites | | --- | --- | | The history no longer reproduces the data | A database rebuilt from empty has the structure but not the movement, so environments diverge | | Applied-state tracking is now yours | Nothing records that the run finished in a given environment; you need your own record | | Two code paths live simultaneously | The read-time fallback exists only for the window, but it is real code with real bugs | | Completion becomes a prerequisite | The constraint that hardens the column, and the deletion of the fallback, cannot ship until the run provably finished everywhere | | Ownership drifts | A run with no named owner stalls, and a stalled run is invisible until something depends on it | The deepest of these is the first. Once data movement lives outside the ordered history, "replay the history" and "reproduce the database" stop being the same statement, and every environment rebuild, every recovery drill, and every new developer's local database inherits the gap. Some teams close it by re-expressing the finished movement as a cheap statement appended to the history afterwards, once the volume is known to be trivial on a fresh database — a fresh build has no legacy rows, so the same movement is often instant there. That is worth doing deliberately rather than discovering the divergence later. ## Making the split finish A split data run is a piece of work with a beginning and an end, and the end is the part that gets lost. Name three things when you approve the split: 1. **A completion criterion** — a query whose result is zero, not a feeling that it is probably done. 2. **A named owner** for the run in each environment, because "the team" does not restart anything. 3. **The follow-up change** that hardens the result and deletes the tolerance code, written as a real ticket at the time of the split, not afterwards. Without them the common outcome is an application permanently written to tolerate a state that was supposed to last a week, and a column that can never be made required because nobody can prove it is fully populated. That is the true cost of the split, and it is why the default for anything small should remain: ship it inside the change, where finishing is not optional.