skip to content

In Debezium, what does the required topic.prefix connector property control?

level: middleimportance: must knowfreq 62%

answer

  1. More than a topic-name decoration
  2. It is also an identity
  3. Offsets and schema history are keyed by it
  4. Renaming it means starting over

basics

~20 s

topic.prefix is the connector's logical source name. Debezium puts it in front of every change topic, names the schema-change, heartbeat and transaction topics with it, and records it in stored state, so it must be unique per connector.

solid answer

~40 s

`topic.prefix` is the logical name Debezium gives a source, and it becomes a namespace. Data-change topics are named `<topic.prefix>.<schema>.<table>` on Postgres, SQL Server and Oracle, and `<topic.prefix>.<database>.<table>` on MySQL. The schema-change topic is the bare prefix, heartbeats go to `__debezium-heartbeat.<topic.prefix>`, transaction metadata to `<topic.prefix>.transaction`, and the prefix also appears in the connector's metrics beans. Crucially it is stamped into the keys of the connector's stored offsets and schema history, so it is an identity, not a label: two connectors sharing a prefix will read each other's state, and changing it on a running connector orphans the old offsets, causing the connector to start over and write to a brand-new set of topics.

code

properties · 4 lines
properties
topic.prefix=inventory
table.include.list=public.customers,public.orders
heartbeat.interval.ms=10000
provide.transaction.metadata=true

go deeper

for a junior

Remember that the prefix is required and that data topics are named prefix, then schema or database, then table. Being able to predict a topic name from a config is the level's expectation.

for a middle

Explain every place the value surfaces — data topics, schema-change topic, heartbeat and transaction topics, metrics — and that it is embedded in the connector's stored position, which is why it must be unique.

for a senior

Be ready to walk through the blast radius of a rename in production: new state key, fresh start, new topics, silently stalled consumers on the old ones, and the migration plan you would run instead.

for a principal

Own the naming convention across teams. Argue for logical, location-free names, an allocation registry so prefixes cannot collide, and a clear rule that topic layout for consumers is solved downstream, not by editing the prefix.

## What the property is `topic.prefix` is a required property on every Debezium connector. It gives the source a *logical name*, chosen by you, that is independent of hostnames or database names. It replaced `database.server.name` in Debezium 2.0; older tutorials and blog posts still show the old key, which is the most common source of confusion when reading examples. The value must be unique among all connectors writing to the same Kafka cluster, and it is validated: only alphanumeric characters, hyphens, dots and underscores are accepted, because the value ends up inside Kafka topic names. ## Where the value shows up **Data-change topics.** One topic per captured table, named by fully-qualified source identifier prefixed with the logical name. For Postgres, SQL Server and Oracle that is `<topic.prefix>.<schemaName>.<tableName>`; for MySQL, which has no schema layer, `<topic.prefix>.<databaseName>.<tableName>`. With `topic.prefix=inventory`, changes to `public.customers` land on `inventory.public.customers`. **Schema-change topic.** Named exactly `<topic.prefix>` — the bare value, no suffix. This surprises people who expect a suffix and then find an unexplained topic sitting next to their data topics. **Heartbeat topic.** `__debezium-heartbeat.<topic.prefix>`, when `heartbeat.interval.ms` is enabled. **Transaction metadata topic.** `<topic.prefix>.transaction`, when `provide.transaction.metadata` is on. **Metrics.** The connector's JMX beans carry the logical name as the `server` attribute, so dashboards and alerts key off it too. Rename the prefix and every metric series starts fresh. **Stored state.** This is the part that turns a naming convention into an identity. The prefix is part of the source-partition key under which Debezium persists its position in the source log, and part of how schema history is scoped. The connector looks up its position using that key on every restart. ## Why uniqueness is not optional Two connectors configured with the same prefix are not merely writing to the same topics — they are reading and writing the same stored state. Each will find the other's recorded position on startup and try to resume from it, on a log it may not even share. On MySQL, both will also try to own the same schema history. The result is not a clean error; it is corrupted position tracking and events that appear or vanish unpredictably. Treat the prefix as a primary key across your whole capture estate, and keep an explicit registry of allocated names if several teams deploy connectors. ## Changing it after the fact Changing `topic.prefix` on a running connector is effectively deleting the connector and creating a new one. On restart it computes a new state key, finds nothing under it, and behaves as a first-time deployment — meaning it will take an initial snapshot if its snapshot configuration says to, or start from the current log position if not. Meanwhile it begins writing to an entirely new set of topics; the old topics stay behind, frozen, and every downstream consumer subscribed to them silently stops receiving data. Nothing is renamed and nothing is migrated. The practical consequence: pick the prefix before you go live, and pick it for what the source *means* rather than where it currently runs. `inventory` survives a database migration from one host to another; `pg-prod-eu-west-1b` does not, and the day you move the database someone will be tempted to rename it. ## Choosing a good value A workable convention is `<domain>` or `<system>_<environment>` — short, stable, meaningful to consumers who will see it in every topic name. Avoid embedding hostnames, IPs, replica identifiers or version numbers. Keep environments separated: staging and production connectors against copies of the same database must not share a prefix, or their state and topics collide. Also remember that the prefix is only the first segment. Downstream teams frequently want a different topic layout — a single topic per aggregate, or a flatter name. That is a routing concern handled by transformations in the pipeline layer, not by bending `topic.prefix`; the prefix should stay a stable identity for the source. ## Interview framing The question usually starts as trivia about topic naming and then pivots: *what happens if you change it?* The strong answer names all three roles at once — namespace for topics, identity for stored offsets and schema history, and label on metrics — and concludes that renaming is a re-deployment, not a rename.

  • A consumer team asks for one topic per aggregate instead of one per table. Where does that belong?
    Not in `topic.prefix`. The prefix is the source's stable identity; re-shaping topic layout is a routing concern handled by transformations in the pipeline layer, or by publishing purpose-built events from the source application. Bending the prefix to satisfy a consumer breaks the connector's state key and every other consumer at the same time.
  • You must move a captured database to a new host. Does the prefix change?
    It should not. The prefix names the logical source, not its location, so the connector keeps its identity, its stored position semantics and its topics across the move. That is exactly why hostnames and regions do not belong in the value — a good prefix survives infrastructure churn.
  • Two connectors are accidentally deployed with the same topic.prefix. How do you recover?
    Stop both, give one a new prefix, and treat it as a fresh deployment: its stored position is untrustworthy because both connectors wrote under the same key. Re-establish its data with a snapshot and point consumers at the new topics. There is no in-place repair, because you cannot tell which position belonged to which connector.

The prefix works like a database schema name that also happens to be the connector's login: it organises everything underneath it, and changing it means you come back as a different user with no history.

saying these in an interview costs you the question

  • Calling topic.prefix a cosmetic naming convention
  • Thinking topics are renamed automatically when the prefix changes
  • Reusing one prefix across staging and production connectors
  • Embedding hostnames or regions in the prefix value
  • Not knowing the schema-change topic is the bare prefix

context