skip to content

When you introduce a schemaVersion field into a live collection, how do you handle documents written before it existed?

level: middleimportance: should knowfreq 42%

answer

  1. You adopt versioning on data that already exists
  2. Absence is itself an encoding
  3. Don't rewrite millions just to add a number
  4. Watch the straggler query, not just the value
  5. Some writer will forget to set it

basics

~20 s

Treat an absent version field as the oldest known generation by convention, rather than rewriting every document just to stamp a number on it. Read code normalizes missing to 1, and migration progress is counted as documents that are missing the field or below the current generation.

solid answer

~50 s

You never get to start a collection versioned; you adopt versioning on data that already exists. The standard move is a convention: **absent means generation 1**. Every read normalizes it in one place — read the field, default it to 1 — so the rest of the code never distinguishes "old" from "unstamped". Rewriting millions of documents purely to add a number is wasted work: if you are going to rewrite them at all, convert their content in the same pass. Two practical consequences. First, every query that finds migration stragglers must match *missing or below current*, not just "below current", or the oldest documents are silently skipped and your zero-count is a lie. Second, check that your store can find missing-field documents efficiently; if it cannot, expect the progress query to be a scan and plan it as a batched sweep rather than a dashboard refresh. And close the tap: make sure no writer still creates documents without the field.

code

javascript · 7 lines
javascript
const CURRENT = 2;

// one place normalizes: no version field means generation 1
const generationOf = (doc) => doc.schemaVersion ?? 1;

// the straggler predicate, shared by backfill and progress metric
const needsMigration = (doc) => generationOf(doc) < CURRENT;

go deeper

for a junior

Remember the convention: a document with no version field is the oldest generation, and read code should default the missing value in one place rather than treating it as an error.

for a middle

Explain why inferring beats stamping — the rewrite cost buys nothing — and why the straggler predicate must match missing as well as below-current, or the count silently lies.

for a senior

Show the operational side: auditing every writer including importers and wholesale replaces, monitoring for newly created unversioned documents, and checking whether your store can find missing-field documents without a scan.

for a principal

Own the adoption plan across services that share the collection — who sets the field, what the normalization contract is, and how the team avoids paying for two full rewrites when one conversion pass would do.

## The situation Versioning is almost never designed in from day one. You have a live collection with millions of documents, none of which carry a version field, and you are about to make the first change that needs one. So the very first migration you run is a migration on data that predates the mechanism you are introducing. ## The convention: absent means the oldest generation The cheap and correct answer is a convention rather than a data change: a document with no version field *is* generation 1. This works because there is only one shape that could have been written before versioning existed — whatever the code was writing at the time. Absence is therefore not ambiguous; it is a perfectly good encoding of "the shape before we started counting". Make the normalization happen once, at the same boundary as the upcasting layer: `const generation = doc.schemaVersion ?? 1`. Above that line, nothing knows that some documents lack the field. If instead each call site handles the missing case, you will find one that forgot, and it will be the one that decides the record is corrupt. ## Why not just stamp them all The alternative — a job that adds `schemaVersion: 1` to every existing document — buys uniformity and nothing else. It costs a full rewrite of the collection, plus the replication and index churn a rewrite implies, to store a number the reader could have inferred for free. Worse, it is usually the wrong pass: if you are prepared to rewrite every document, convert them to the current shape in that same pass and skip generation 1 entirely. Rewriting twice — once to stamp, once to convert — doubles the most expensive part of the work. Stamping is defensible in one narrow case: when your store makes finding documents without a field far more expensive than finding documents with a low value, and you expect to run progress queries constantly. Even then, weigh it against just running the real conversion. ## Finding the stragglers This is where teams get burned. Once the convention is in place, the set of documents that still need migrating is *missing the field, or holding a value below the current generation*. A progress query that tests only "version is below current" matches none of the pre-versioning documents at all, because they have no value to compare. The result is a comforting zero, an old read branch deleted on the strength of it, and a production error the first time a genuinely ancient document is loaded. Write the straggler predicate once, in the same module as the normalization, and use it for both the backfill's batch selection and the progress metric so the two can never disagree. The second trap is efficiency. Locating documents that lack a field is not always something a store can satisfy from an index — that depends on the product and the index options in use. Check it for your store before you promise anyone a live progress dashboard. If the answer is "this is a scan", treat the sweep as a batched background job that reports as it goes, rather than a query you run repeatedly at speed. ## Closing the tap The convention only holds if the set of unversioned documents is finite and shrinking. That means every write path must set the field before you start counting: the main application, but also importers, admin tooling, seed and fixture scripts, sibling services that write the same collection, and any bulk operation that replaces whole documents rather than editing fields — a wholesale replace built from a partially populated object drops the version silently and quietly manufactures new "legacy" documents long after you thought the era ended. A useful guard is a monitor on documents lacking the field that were created recently. If that count is not zero, some writer has not been updated, and no amount of backfilling will ever converge. ## Interaction with the first real change Because your first version bump lands on unversioned data, sequence it deliberately. Ship the reader that normalizes absent to 1 and understands generation 2, and only then ship writers that produce generation 2. From that moment the collection contains exactly two populations — unstamped generation 1, and stamped generation 2 — and the backfill's job is to empty the first. After that run, the convention is still worth keeping in the reader as a permanent safety net; it costs one null-coalescing operator and protects you from any writer that ever slips through unversioned.

  • What goes wrong if the migration progress query tests only 'version below current'?
    It matches no pre-versioning document, because those have no value to compare against, so the oldest and most fragile records are invisible. The query returns zero, someone deletes the old read branch on that evidence, and the first ancient document loaded afterwards fails in production. The predicate must be 'field missing or value below current', written once and shared by the backfill and the metric.
  • When is it worth running a job purely to stamp existing documents with generation 1?
    Rarely. It rewrites the whole collection to store a value the reader can infer for free, and if you are willing to rewrite everything you should convert content in the same pass instead. The narrow case is a store where finding documents lacking a field is far more expensive than comparing a stored value and you need frequent, cheap progress queries.
  • How do you stop new unversioned documents appearing after adoption?
    Audit every writer, not just the main application: importers, admin tools, seed scripts, sibling services and any operation that replaces a whole document rather than editing fields, since a wholesale replace built from a partial object silently drops the field. Then monitor for recently created documents that lack it; a non-zero count means a writer was missed and the population will never converge.

saying these in an interview costs you the question

  • Rewrites the whole collection just to add a version number
  • Straggler query checks only 'below current', missing unstamped documents
  • Treats a document with no version field as corrupt
  • Assumes only the main application writes to the collection
  • Handles the missing-field default at every call site instead of once

context

resolving links…