How do you verify that saved user data survives both an upgrade and a rollback to the previous release?

### The two directions Stored data outlives the build that wrote it, so a release has to be checked in both directions. - **Backward compatibility** — the *new* build opens data written by an *older* one. This is the upgrade path, and most teams test at least the happy version of it. - **Forward compatibility** — the *older* build opens data written by the *newer* one. This is the rollback path, and it is the one that gets skipped, because nobody plans to roll back. Yet rollback is the standard incident response, and a rollback that loses data is worse than the incident. When an interviewer asks about upgrade testing, naming the second direction unprompted is most of the answer. ### Building the corpus The hardest input is realistic saved data. A file written by the current build with a script is not evidence, because it lacks everything old builds used to produce. Keep a **versioned corpus**: for every release still in the support matrix, saved files, local databases and preference stores produced by that release doing real work — including the ugly cases: a half-finished record, a store left behind by a crash, data written in a non-default locale, a store at realistic size rather than three rows. Freeze each artefact and keep it under version control with the release that produced it. When a release drops out of the matrix, its corpus entry retires with it. ### The upgrade matrix Users do not upgrade one version at a time. The paths worth running are: the immediately previous release; the oldest release still supported; at least one skipped-version jump; and an upgrade interrupted partway and then resumed. For each, assert **content** — record counts, specific field values, referential integrity, ordering, and anything derived — not merely that the application starts. A migration that silently drops a column passes a start-up check every time. Also time it. Migration duration on the largest realistic dataset is a release-gating number, because during a migration the product is unavailable. ### The rollback drill The rollback drill is the one to describe in detail: 1. Take a snapshot of the stored data before the upgrade. 2. Upgrade, then exercise the product so the new build **writes** — this is essential, because forward compatibility is only tested against data the new build produced. 3. Install the previous release over that data. 4. Assert what the old build does: does it open at all, does it discard fields it does not recognise, does it round-trip data it cannot interpret, does it refuse on a version stamp? The usual designs that make step 4 survivable are a **version stamp** in the stored data plus a rule about what an older reader does with unknown content — either preserve it untouched on rewrite, or refuse to write at all and stay read-only. Both are defensible; silently dropping it is not. Where the new format genuinely cannot be read by the old build, the fallback is the pre-upgrade snapshot, and then the test is that restoring it actually works and its age is acceptable. ### A worked example A hotel booking channel manager ships an on-premises connector that keeps a local store of rate plans and pending updates, peaking near 1,200 requests per minute during promotions. Release N added a per-plan restriction field to the store. A property hit an unrelated problem and rolled back to N-1 the same evening; N-1 read the store, ignored the restriction field, and rewrote every record without it. The connector then pushed the pre-existing restriction-free rates from its now-authoritative local store — a **stale-cache read** promoted into a write — and 41 rooms went on sale at an unrestricted rate for about 3 hours 20 minutes. Nothing about that was a code defect in the usual sense: the upgrade worked, the rollback worked, and the data was destroyed between them. The corrective work was a preserve-unknown-content rule for the store's reader, a version stamp read by both builds, a pre-upgrade snapshot taken automatically, and a rollback drill added to the release checklist that exercises exactly the sequence above. ### What to say about coordination When the same data is written by more than one participant during a rolling upgrade, the honest answer includes a transition window: for one release both builds must be able to read and write the format, so a rollback never meets data it cannot handle. That means format changes land at least one release before the code that depends on them — add-and-tolerate first, require later, remove later still. It is slower, and it is what makes rollback a routine action instead of a gamble.