skip to content

What does the transitive form of a compatibility guarantee quantify over that the plain form does not?

level: seniorimportance: nice to knowfreq 30%

answer

  1. how far back does the claim reach?
  2. per-step guarantees do not compose
  3. quantify over history, not one hop
  4. retained data outlives one upgrade
  5. each direction has its own transitive form

basics

~20 s

Transitive compatibility quantifies over every earlier version, not just the immediately preceding one. Plain per-step checks do not compose: version 3 can pass against version 2, and version 2 against version 1, while a version 3 reader still fails on version 1 data.

solid answer

~50 s

The plain form compares the new version against **one** predecessor. The transitive form asserts the same property against **every** earlier version. The distinction matters because the per-step property is not transitive. Take a field that does not exist in version 1, is added in version 2 with a declared default, and becomes required in version 3. A version 2 reader over version 1 data supplies the default, so that step passes; a version 3 reader over version 2 data finds the field present, so that step passes too; but a version 3 reader over version 1 data has no value and no default, and fails. Both per-step checks were green and the two-step claim is false. You need the transitive form wherever data outlives one release — a long-retention log, an archive, a consumer two versions behind — and the plain form is enough where nothing older than one version can still reach a reader.

code

pseudocode · 11 lines
pseudocode
// plain check: the new version against its immediate predecessor only
function backward_ok(new_version, history):
    previous = last(history)
    return reader(new_version) decodes data_written_under(previous)

// transitive check: against every version whose data can still be read
function backward_ok_transitive(new_version, history):
    for each past_version in history:
        if not reader(new_version) decodes data_written_under(past_version):
            return false
    return true

go deeper

for a junior

Know that a compatibility claim covers a specific range of versions. "Compatible" with no range attached is not yet a statement anyone can verify.

for a middle

Explain why two green per-step checks do not imply the two-step claim, using a field whose default is introduced in one version and removed in a later one.

for a senior

Tie the choice to the channel: retained logs, archives, backfills and stragglers two versions behind are what force the transitive form, and a drained short-lived channel is what makes the weaker one honest.

for a principal

Decide and write down how far back readers are required to reach on each channel, and recognise that an unwritten bound is one nobody is maintaining.

## Two claims that sound alike Every compatibility direction comes in two strengths, and interviews rarely get past the weaker one. - **Plain (one-step):** the new version satisfies the direction against its immediate predecessor. - **Transitive:** the new version satisfies the direction against **every** earlier version in the history. The transitive form is strictly stronger, and it does not follow from repeated application of the weaker one. That is the whole content of the distinction, and it is the part candidates get wrong: a chain of green per-step checks feels like it should compose, and it does not. ## A worked counterexample, walked version by version Consider the backward direction — a newer reader over older data — and a field that changes status across three versions: 1. **Version 1** does not contain the field at all. 2. **Version 2** introduces it, with a declared default for readers that do not find it. 3. **Version 3** makes it required, and removes the default. Now walk the readers over the data: - A **version 2 reader over version 1 data**: the field is absent, the default applies, the record decodes. The one-step check passes. - A **version 3 reader over version 2 data**: every version 2 writer emitted the field, so it is present, and the record decodes. The one-step check passes. - A **version 3 reader over version 1 data**: the field is absent, there is no longer a default to fall back on, and the decode has nothing to produce. The two-step claim **fails**. Two green checks, one red conclusion. Nothing in the per-step procedure was wrong; it simply never asked the question that eventually got asked in production. ## Where the difference actually bites The transitive form is the one you need whenever bytes can outlive more than one release: - A **retained log** replayed from its beginning by readers on the current version. The oldest records may be many versions old, and the reader is asked about all of them at once. - **Archived files** read months or years later by whatever version is current then. - A **consumer stuck two or more versions behind**, which is normal in a fleet of a few hundred consumers where some teams deploy quarterly. - **Reprocessing or backfill**, which deliberately points today's code at the full history. The plain form is genuinely sufficient in the opposite conditions: a channel drained and restarted between releases, or a synchronous hop where every message is decoded seconds after it is produced. Nothing older than one version is ever presented to a reader there, so the extra strength constrains schema authors without buying anything. ## Both directions have both strengths The ladder is per direction, and the four rungs are independent claims: | Claim | What it asserts | |---|---| | Backward | the new reader handles the previous version's data | | Backward transitive | the new reader handles **every** earlier version's data | | Forward | the previous version's reader handles the new data | | Forward transitive | **every** earlier version's reader handles the new data | Full compatibility is the conjunction of the two plain forms; full transitive is the conjunction of the two transitive forms, and it is the strongest of the set. Forward transitive is the hardest to sustain in practice, because it constrains the new version against readers written before anyone knew the new version would exist. ## How to talk about it without overclaiming Three habits keep the claim honest: 1. State the **range** the claim covers: "backward compatible with everything from version 4 onwards" is checkable; "backward compatible" is not. 2. Tie the range to the **retention** of the channel rather than to the schema. The question "how far back can a reader be asked to reach?" is answered by how long data and stragglers live, not by the schema's design. 3. Where the range is bounded deliberately — history older than some version is no longer read — say so explicitly, because that bound is the thing that makes the weaker check adequate. A bound nobody has written down is a bound nobody is maintaining.

  • When is the plain, per-step check genuinely enough?
    When nothing older than the previous version can still reach a reader: a short-retention channel fully drained between releases, or a synchronous hop where every message is decoded within seconds. As soon as records or stragglers outlive one upgrade cycle, the per-step check stops describing the pairings that actually occur.
  • Does the transitive variant exist for the forward direction too?
    Yes. Transitive forward means readers on every earlier version can decode data written under the new one, and transitive full is both directions over the whole history. Forward transitive is the hardest to hold, because it constrains the new version against readers that were written before it was conceived.

saying these in an interview costs you the question

  • Assumes per-step compatibility composes across several versions
  • Thinks checking against the previous version covers all history
  • Ignores retention when choosing between the two strengths
  • Believes only the backward direction has a transitive form
  • Claims compatibility without naming the version range covered