skip to content

At startup, what does KafkaAdmin do when a declared NewTopic already exists on the broker but with different partitions, replication factor, or configs?

level: seniorimportance: should knowfreq 28%

answer

  1. increase partitions only, never shrink
  2. RF change ignored/logged
  3. configs changed only if modifyTopicConfigs=true
  4. createPartitions vs incrementalAlterConfigs
  5. growing partitions breaks key ordering

basics

~20 s

It won't recreate the topic. It can increase partitions to match a higher declared count (never decrease). It ignores replication-factor differences. Topic configs are only altered if modifyTopicConfigs is enabled; otherwise differences are left alone.

solid answer

~40 s

KafkaAdmin reconciles existing topics conservatively. If the NewTopic declares *more* partitions than the topic currently has, KafkaAdmin issues a createPartitions call to grow it — but partition count can never shrink (Kafka forbids it), so a lower declared number is ignored. Replication-factor changes on an existing topic are **not** performed (changing RF requires a reassignment; KafkaAdmin just logs and moves on). Topic-level configs are only altered when `setModifyTopicConfigs(true)` (spring.kafka.admin.modify-topic-configs) is set; then KafkaAdmin diffs and applies changed configs via incrementalAlterConfigs. Without it, config drift is silently tolerated. Increasing partitions is itself risky: it changes key→partition routing and can break per-key ordering guarantees, so it shouldn't be treated as a free operation.

code

java · 15 lines
java
@Bean
KafkaAdmin.NewTopics topics() {
    return new KafkaAdmin.NewTopics(
        TopicBuilder.name("orders").partitions(12).replicas(3)
            .config(TopicConfig.RETENTION_MS_CONFIG, "604800000").build()
    );
}

// Opt in to config reconciliation on existing topics (Boot equivalent:
// spring.kafka.admin.modify-topic-configs=true). Without this, a changed
// RETENTION_MS on an already-existing 'orders' topic is ignored.
@Bean
KafkaAdmin.KafkaAdminCustomizer modifyConfigs() {
    return admin -> admin.setModifyTopicConfigs(true);
}

go deeper

for a junior

Know it won't recreate an existing topic.

for a middle

Know partitions can only increase and configs need modifyTopicConfigs to change.

for a senior

Explain RF is never changed, the incrementalAlterConfigs path, and the ordering risk of growing partitions.

for a principal

Decide when KafkaAdmin's additive reconciliation is insufficient and move to dedicated topic-management tooling.

## Reconciliation, not recreation KafkaAdmin never deletes and recreates an existing topic. On startup it compares each declared `NewTopic` against the live topic and applies only *safe, additive* changes. ### Partitions - Declared partitions **>** actual ⇒ KafkaAdmin calls **`createPartitions`** (AdminClient) to increase the count to the declared number. - Declared partitions **≤** actual ⇒ no-op. Kafka **cannot reduce** partition count, so a lower number is simply ignored (no error). - **Why this matters:** adding partitions changes the modulo used in default key hashing (`hash(key) % partitions`). Existing keys can start landing in different partitions, which **breaks per-key ordering** and can disrupt stateful consumers/compaction assumptions. "Auto-grow partitions" is not free. ### Replication factor - KafkaAdmin does **not** change the replication factor of an existing topic. RF changes require a partition reassignment plan (`kafka-reassign-partitions` / AlterPartitionReassignments), which KafkaAdmin does not perform. A mismatch is logged and ignored. ### Topic configs (retention, cleanup.policy, etc.) - Default: **`modifyTopicConfigs=false`** ⇒ config differences on an existing topic are ignored (no changes). - Set **`setModifyTopicConfigs(true)`** (Boot: `spring.kafka.admin.modify-topic-configs=true`) ⇒ KafkaAdmin fetches current configs, diffs them against the NewTopic's configs, and applies the changed ones via **`incrementalAlterConfigs`**. It only touches keys you declared. ## Behavior summary table | Change | Applied on existing topic? | |---|---| | Increase partitions | Yes (createPartitions) | | Decrease partitions | No (impossible in Kafka; ignored) | | Change replication factor | No (logged, ignored) | | Change topic configs | Only if modifyTopicConfigs=true | | Recreate / delete | Never | ## Practical guidance - Treat KafkaAdmin as **create-or-additively-reconcile**, not a full desired-state controller. It won't fight you, but it also won't fully converge (RF, config drift by default, shrink). - Enabling `modifyTopicConfigs` is convenient but means a redeploy can silently alter production retention/cleanup — review those beans carefully. - If you need real desired-state topic management (RF changes, deletions), use dedicated tooling (Terraform Kafka provider, `kafka-topics.sh`, GitOps) rather than relying on KafkaAdmin.

  • Why can increasing partitions on a live topic be dangerous even though KafkaAdmin allows it?
    Default partitioning routes by hash(key) % partitionCount. Adding partitions changes that modulo, so a given key may move to a new partition. Per-key ordering is only guaranteed within a partition, so history for a key can now span two partitions — breaking ordering and any stateful/compaction assumptions.
  • How would you actually change a topic's replication factor?
    Not via KafkaAdmin — it doesn't do RF changes. You run a partition reassignment (kafka-reassign-partitions.sh / AlterPartitionReassignments API) or use infra tooling, which is broker/cluster operations territory.

saying these in an interview costs you the question

  • Claiming KafkaAdmin recreates or deletes a topic to match the bean
  • Thinking replication factor is updated automatically on redeploy
  • Assuming topic configs always reconcile (they don't unless modifyTopicConfigs=true)
  • Believing partitions can be decreased

context

resolving links…