What is partition-count and config drift between a source and a mirrored Kafka cluster, why is it dangerous, and how does MirrorMaker 2 try to keep them in sync?

**What 'drift' means.** When MirrorMaker 2 (MM2) mirrors a topic, the target copy should structurally match the source. *Drift* is any divergence between them: most importantly the **partition count**, but also **topic-level configuration** such as `retention.ms`, `cleanup.policy` (delete vs compact), `max.message.bytes`, `min.insync.replicas`, and `segment.bytes`. **Why partition-count drift is dangerous.** A Kafka *partition* is the unit of ordering and parallelism; a topic with N partitions is N ordered logs. When a producer sends a record with a key, the default partitioner computes `partition = hash(key) % numPartitions`. So which partition a key lands in depends on the *partition count*. If the source has 12 partitions and the target has 6, then after a DR *failover* (clients now produce to the target) the same key `user-42` hashes to a different partition number than it did on the source. Consequences: (1) **per-key ordering breaks** — a consumer that assumed all records for a key are in one partition now sees them split or interleaved across the source-era and target-era histories; (2) **stateful/co-partitioned joins** (Kafka Streams) that rely on matching partition counts across topics fail or rebalance incorrectly; (3) **offset/position assumptions** about specific partitions become invalid. This is a *silent correctness* bug, not a crash, which makes it worse. **Why config drift is dangerous.** If `retention.ms` differs, data expires at different times on each cluster — the DR copy may have already deleted records you assumed were safe, blowing your RPO assumptions. If `cleanup.policy` differs (source compacted, target delete), the log *content* diverges. If `max.message.bytes` is smaller on the target, large records that the source accepted get rejected on the replica, silently dropping data. **How MM2 reconciles.** The `MirrorSourceConnector` is responsible for keeping topics aligned: - **Auto-create + partition propagation:** it creates missing target topics and, on a refresh interval, detects when the *source* partition count grew and adds partitions to the target so counts match. (It does NOT shrink partitions — Kafka can't reduce partition count anyway.) - **Config sync:** when `sync.topic.configs.enabled=true` (default), it periodically copies allowed source topic configs to the target. A default exclusion list / `config.properties.exclude` keeps replication-specific or broker-managed configs from being overwritten (e.g. you don't blindly copy `min.insync.replicas` if topologies differ). - **Refresh intervals:** `refresh.topics.interval.seconds` and `sync.topic.configs.interval.seconds` (names vary slightly by version) control how often reconciliation runs — so there is always a window where manual target changes cause temporary drift until the next cycle. **Edge cases and pitfalls.** - If someone manually adds partitions on the *target*, MM2 won't remove them, and now hashing differs permanently — never edit mirrored topics by hand. - Excluded configs (by design) can drift legitimately; know which ones MM2 ignores. - Increasing partitions on the source *also* breaks key->partition mapping going forward (a general Kafka caveat), and MM2 faithfully propagates that — so partition increases should be rare and deliberate on both sides. - ACLs and quotas are generally NOT synced by MM2 (separate concern), another form of operational drift.