Which topics does MM2 deliberately NOT replicate as ordinary data, and how does it identify them?
answer
- __consumer_offsets, __transaction_state excluded
- MM2 heartbeats / checkpoints / offset-syncs
- isInternalTopic() + DefaultTopicFilter
- topics.exclude default kills internals
- offsets moved via checkpoints, not raw topic
basics
~20 sMM2 skips internal and system topics: Kafka's own __consumer_offsets and transaction-state topics, and MM2's own heartbeats, checkpoints, and offset-sync topics. It identifies them via the ReplicationPolicy's isInternalTopic check and the topic filter, so they aren't replicated as user data.
solid answer
~40 sMM2 must not blindly copy every topic, or it would replicate Kafka's internal machinery and its own bookkeeping topics, corrupting the destination. It excludes: Kafka system topics (`__consumer_offsets`, `__transaction_state`, and other `__`-prefixed internals); MM2's own management topics — `heartbeats`, `<source>.checkpoints.internal`, and the offset-sync topic (`mm2-offset-syncs.<cluster>.internal`); and topics matched by exclusion filters. Identification happens via two mechanisms: the `ReplicationPolicy.isInternalTopic(name)` method (which by default flags `__`-prefixed and MM2's internal names), and the `topic.filter.class` (default `DefaultTopicFilter`) combined with the `topics` allowlist and `topics.exclude` denylist. The default exclude pattern already removes internal topics. This filtering is essential: replicating `__consumer_offsets` directly would clobber the destination's own consumer-group state, which is instead handled separately by MM2's offset-sync/checkpoint mechanism.
go deeper
Know internal/system topics like __consumer_offsets aren't copied as data.
List MM2's own internal topics and that a filter excludes them by default.
Explain isInternalTopic plus DefaultTopicFilter/topics.exclude, and that offsets move via checkpoints not the raw offsets topic.
Reason about filter/isInternalTopic interplay, custom-policy obligations, and failure modes of misconfigured exclusions.
## Why filtering is necessary A Kafka cluster hosts more than user data topics. It has internal/system topics that store cluster state. MM2 also creates its own bookkeeping topics. If MM2 replicated all of these as ordinary data, the destination's own state would be overwritten or duplicated — e.g. copying the source's `__consumer_offsets` onto the destination would clobber the destination's consumer-group positions. So MM2 deliberately filters them out. ## What gets excluded 1. **Kafka system topics**: `__consumer_offsets` (consumer-group offsets), `__transaction_state` (transactional producer state), and other `__`-prefixed internals (e.g. cluster metadata topics). 2. **MM2's own management topics**: - `heartbeats` — emitted by `MirrorHeartbeatConnector` to measure liveness/lag. - checkpoint topic — `<sourceAlias>.checkpoints.internal`, written by `MirrorCheckpointConnector`, storing translated consumer offsets. - offset-sync topic — `mm2-offset-syncs.<targetCluster>.internal`, mapping source↔destination offsets for failover. 3. **Explicitly excluded topics** — anything matched by `topics.exclude` or not matched by the `topics` allowlist. ## How MM2 identifies them — two cooperating mechanisms ### a) ReplicationPolicy.isInternalTopic(name) The policy exposes `isInternalTopic(String topic)`. `DefaultReplicationPolicy`'s implementation returns true for names that are MM2-internal or begin with the internal prefix (`__`, and the MM2 `.internal` suffixed names). MM2 consults this so internal topics aren't treated as replicable user data — and so a remote name's parsing doesn't misfire on them. ### b) Topic filter `topic.filter.class` (default `org.apache.kafka.connect.mirror.DefaultTopicFilter`) decides eligibility from: - `topics` — allowlist regex (default `.*`). - `topics.exclude` (formerly `topics.blacklist`) — denylist regex; its default already excludes internal/MM2 topics (patterns like `.*[\-\.]internal`, `.*\.replica`, `__.*`, plus MM2's heartbeats/checkpoints/offset-syncs). Both must pass: a topic is replicated only if it's not internal and clears the filter. ## Consequences if you get it wrong - Loosening `topics.exclude` to include `__consumer_offsets` would overwrite the destination's offsets — consumer groups jump to wrong positions. - Replicating MM2's own `heartbeats`/`checkpoints` could create feedback and confuse monitoring. ## How offsets ARE moved (the right way) Consumer offsets are NOT moved by replicating `__consumer_offsets`. Instead `MirrorCheckpointConnector` reads source group offsets, translates them to destination offsets using the offset-sync topic, and writes `checkpoints`; clients use `RemoteClusterUtils`/the checkpoint topic to resume at the correct position on failover. This is why the raw internal offsets topic must stay excluded. ## Edge cases - Custom `ReplicationPolicy` must implement `isInternalTopic` correctly, including any custom internal names, or filtering leaks. - A user topic that happens to start with `__` would be excluded by the default pattern — rename or adjust filters if that's unintended.
- Why must MM2 NOT replicate __consumer_offsets directly?It stores the destination's own consumer-group positions; copying the source's over it would clobber destination offsets. Offsets are instead translated via the offset-sync and checkpoint topics so consumers resume at correct positions on failover.
- What two mechanisms together decide a topic is excluded?ReplicationPolicy.isInternalTopic (flags __-prefixed and MM2-internal names) and the topic filter (DefaultTopicFilter with the topics allowlist and topics.exclude denylist, whose default already excludes internal/MM2 topics).
saying these in an interview costs you the question
- Saying MM2 replicates __consumer_offsets to move consumer positions (it uses checkpoints/offset-sync instead)
- Forgetting MM2's own heartbeats/checkpoints/offset-sync topics are also excluded
- Thinking only the filter matters and ignoring isInternalTopic