A captured table gains a nullable column; what do Debezium's Avro consumers and the schema registry see?
answer
- the table model updates, then the value schema
- a nullable column becomes an optional field
- optional plus default keeps old readers working
- the reverse direction is what fails
- the task dies rather than dropping a record
basics
~20 sDebezium emits a DDL change event and then change events whose value schema carries the new optional field, so a new schema version is registered. Adding a nullable column is compatible; dropping or retyping a column is what stops the pipeline.
solid answer
~50 sWhen the source table changes, Debezium updates its internal table model and the next change event carries a new value schema. With an Avro converter and a schema registry, that schema is registered as a new version of the topic's subject. A nullable column becomes an optional field with a default of null, which satisfies backward compatibility, so consumers still running the old schema keep working and simply do not see the field until they are rebuilt. The painful cases are the other direction: dropping a column, narrowing a type, or renaming — the registry rejects the incompatible registration and the connector task fails, halting capture for the whole table. If `include.schema.changes` is on, the DDL also lands on the schema change topic named after `topic.prefix`, which is the signal downstream systems should act on. The practical rule is additive-and-nullable in the source, coordinated deploys for anything else.
code
sql · 5 lines-- compatible: emitted as an optional field defaulting to null
ALTER TABLE orders ADD COLUMN promo_code VARCHAR(32) NULL;
-- breaking: readers that require the field can no longer decode
ALTER TABLE orders DROP COLUMN legacy_ref;go deeper
Know that changing a captured table changes the shape of the events Debezium publishes, and that adding a nullable column is the safe kind of change while removing one is not.
Explain why a nullable column becomes an optional field with a default, and why that specific property is what lets a consumer built on the previous schema keep reading new records unchanged.
Describe the failure precisely: an incompatible registration fails the task and halts capture for that table, lag accumulates, and finite source log retention can turn the stall into a forced re-snapshot.
Decide whether an incompatible source change should stop the pipeline or flow through with quarantine, and put the compatibility gate in the producing team's build so the cost lands where the change was made.
## The chain of events An `ALTER TABLE ... ADD COLUMN` in the source produces four visible effects, in order. First, the connector observes the DDL — from the binlog for MySQL, from refreshed relation metadata for Postgres — and updates its in-memory table model. Second, if schema-change publishing is enabled, a DDL change event is written to the schema change topic named after the connector's `topic.prefix`; this is the feed a downstream system watches to auto-evolve a target table or to raise an alert. Third, the next row change from that table is serialised against a new value schema containing the new field. Fourth, with a registry-backed converter, that schema is registered under the topic's subject, becoming a new version. ## Why adding a nullable column is the easy case Debezium maps a nullable source column to an optional field whose default is null. In Avro terms the new field is a union with null and it has a default, which is precisely the condition under which a reader using the *old* schema can still read data written with the new one, and a reader using the new schema can read *old* data by filling in the default. That is why teams get away with additive schema changes for years without touching consumers: the new field is simply invisible to anyone who has not redeployed. Note that both the `before` and `after` structs gain the field if you publish the full envelope, and that snapshot events use the same schema as streamed ones. If you have flattened events, the row schema gains the field directly. ## Where it breaks **Dropping a column.** Consumers that require the field can no longer read new records, and under a backward-compatibility rule the registry refuses the new schema. The connector task fails and capture for that table stops — this is the outage, and it is caused by a source-side migration that nobody told the data team about. **Changing a type.** Widening an integer or lengthening a varchar is often tolerable; changing an integer column to text, or a timestamp's precision, generally is not, and Debezium's temporal and decimal handling adds its own wrinkles because the wire representation depends on connector settings. **Renaming.** A rename is a drop plus an add as far as the schema is concerned, with the extra insult that the data moves silently into a field nobody is reading. **Non-nullable additions.** A `NOT NULL` column with no default has no safe default in the emitted schema, so old readers have nothing to fall back on. **Illegal names.** Avro's naming rules are stricter than SQL's. A table or column whose name is legal in the database but not in Avro must be adjusted, which Debezium handles through a name-adjustment mode; leaving it unset and then hitting such a name is a startup-time surprise rather than a runtime one. ## Failure shape and blast radius The failure is not a dropped record. It is a failed task: capture for the table stops, lag climbs, and on engines with finite log retention a long enough stall becomes a mandatory re-snapshot. That asymmetry is the argument for treating source DDL as a governed change rather than an ordinary migration. ## How mature teams run this Make additive-and-nullable the default migration shape, and require a two-phase deploy for anything else: add the new column, backfill, move consumers, then drop the old column in a later release once nothing reads it. Consume the schema change topic and alert on it, so a data team learns about a source migration when it happens rather than when the pipeline dies. Put a compatibility check in the producer team's CI so an incompatible migration fails a build instead of a connector. And decide deliberately whether you want the pipeline to *stop* on an incompatible change — which is what a strict registry rule gives you and is often the right answer for a system of record — or to keep flowing with a permissive landing schema, which is a different discipline entirely. The deepest version of this answer is the one that says the outbox pattern sidesteps most of it: when the published payload is a designed event rather than a row image, an `ALTER TABLE` on a business table changes nothing that any consumer can see, and schema evolution becomes a versioning decision made deliberately by the producing team rather than a side effect of a migration.
- Why is a column rename more dangerous than an add followed by a drop?Because it is both at once, with no window in between. Consumers lose the old field and gain an unread new one in the same release, and any incompatibility rule rejects the schema immediately. The safe form is explicitly staged: add the new column, backfill it, migrate consumers, then drop the old one in a later deploy.
- What should a downstream system do with the schema change topic?Consume it as an operational signal. Typical uses are auto-evolving a target table with the added column, opening a ticket or paging the owning team on a destructive change, and recording a timeline that explains why a load's shape shifted. Ignoring it means the first sign of a source migration is a failed connector task.
- How does routing events through an outbox change this picture?It removes the coupling. Consumers see a payload the service authored, not a row image, so an ALTER TABLE on a business table is invisible to them. Evolution becomes a deliberate act by the producing team — a new event version — instead of an accident of a migration, at the cost of the service having to maintain that mapping.
saying these in an interview costs you the question
- Says Debezium keeps emitting the old schema until the connector is restarted
- Assumes any schema change is safe because the registry will sort it out
- Treats a column drop as equivalent in risk to a column add
- Thinks an incompatible change loses a few records rather than halting capture
- Confuses the public schema change topic with the connector's internal history