skip to content

KafkaAdmin & Topic Provisioning

2 roadmaps5 questions

Declaring NewTopic beans lets the application create topics with the right partitions, replicas and configs at startup via AdminClient. Interviewers ask whether apps should create their own topics at all, which is a genuine ops debate.

part ofSpring Frameworkoverview, primer and where to startread it →
on this pageshow

explore

questions

5

How do you make Spring create a Kafka topic automatically when the application starts?

level: juniorimportance: must knowfreq 45%

answer

  1. NewTopic @Bean
  2. KafkaAdmin scans context at startup
  3. TopicBuilder.name().partitions().replicas()
  4. wraps AdminClient
  5. not broker auto.create.topics.enable

basics

~10 s

Declare a NewTopic as a @Bean (usually built with TopicBuilder). Spring's KafkaAdmin bean scans the context for NewTopic beans at startup and creates any that don't yet exist on the broker.

solid answer

~40 s

Spring for Apache Kafka provides a KafkaAdmin bean (auto-configured by Spring Boot from spring.kafka.bootstrap-servers). At context startup KafkaAdmin looks through the ApplicationContext for every NewTopic bean and, using an internal Kafka AdminClient, creates the ones that are missing on the broker. You just expose a NewTopic @Bean, most idiomatically via TopicBuilder: TopicBuilder.name("orders").partitions(6).replicas(3).build(). Nothing else is required — no explicit AdminClient calls, no imperative creation code. If the topic already exists it is left in place (Spring may add partitions but never fails just because it's there). This is declarative, in-app provisioning; it is not the broker-side auto.create.topics.enable feature, which is a completely separate cluster setting.

code

java · 13 lines
java
@Configuration
class TopicConfig {

    // KafkaAdmin (auto-configured by Spring Boot) finds this bean at
    // startup and creates the topic if it does not already exist.
    @Bean
    NewTopic ordersTopic() {
        return TopicBuilder.name("orders")
                .partitions(6)
                .replicas(3)
                .build();
    }
}

go deeper

for a junior

Know that a NewTopic @Bean plus KafkaAdmin means Spring creates the topic at startup.

for a middle

Explain that KafkaAdmin wraps AdminClient and scans the context; know TopicBuilder and the 1/1 default.

for a senior

Contrast with broker-side auto-create and reason about idempotency and existing-topic behavior.

for a principal

Weigh in-app declarative provisioning against infra-as-code and environment-specific replication.

## The pieces **KafkaAdmin** (`org.springframework.kafka.core.KafkaAdmin`) is a Spring-managed bean that wraps the Kafka client's **AdminClient** (`org.apache.kafka.clients.admin.AdminClient`). AdminClient is the low-level Kafka API for administrative operations (create/delete topics, alter configs, list topics). KafkaAdmin's job is to make topic creation *declarative* so you don't call AdminClient by hand. With **Spring Boot**, `KafkaAutoConfiguration` creates a `KafkaAdmin` for you from `spring.kafka.bootstrap-servers` and any `spring.kafka.admin.*` properties. Without Boot you declare it yourself: ```java @Bean KafkaAdmin kafkaAdmin() { return new KafkaAdmin(Map.of(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092")); } ``` **NewTopic** (`org.apache.kafka.clients.admin.NewTopic`) is a plain description of a topic: its name, partition count, replication factor, and topic-level configs. You never create the topic yourself — you just *describe* it as a Spring bean. **TopicBuilder** (`org.springframework.kafka.config.TopicBuilder`) is a fluent helper that produces a `NewTopic`. ## How auto-creation fires During context initialization, KafkaAdmin (via its `initialize()` method) collects **all `NewTopic` (and `KafkaAdmin.NewTopics`) beans** in the context and calls AdminClient to create the missing ones. This happens once, at startup. Topics that already exist are not re-created. ## Minimal example ```java @Configuration class TopicConfig { @Bean NewTopic ordersTopic() { return TopicBuilder.name("orders") .partitions(6) .replicas(3) .build(); } } ``` ## Key gotchas - **Only `NewTopic` *beans* are provisioned.** A `NewTopic` you `new` inside a method but never expose as a bean does nothing. - If no partitions/replicas are set, TopicBuilder defaults to **1 partition, 1 replica** — a classic footgun in multi-broker/prod clusters. - This is *not* `auto.create.topics.enable`. That is a **broker** setting that lazily creates a topic the first time any client references an unknown one, with broker-default partitions/replication. KafkaAdmin auto-creation is explicit, in-application, and gives you control over partitions/replicas/configs. - If the broker is unreachable at startup, by default the app still starts (creation just fails/logs) — it is not fatal unless you opt in.

  • What is the difference between this and the broker's auto.create.topics.enable?
    KafkaAdmin auto-creation is explicit and in-app: you declare NewTopic beans with chosen partitions/replicas/configs, created at startup. auto.create.topics.enable is a broker cluster setting that lazily creates a topic on first client reference using broker defaults, with no per-topic control — and it's out of the application's hands.
  • If you declare a NewTopic but never mark it @Bean, what happens?
    Nothing. KafkaAdmin only scans the ApplicationContext for NewTopic beans, so a NewTopic that isn't a Spring bean is invisible to provisioning.

saying these in an interview costs you the question

  • Thinking the topic is created lazily on first send rather than at context startup
  • Confusing KafkaAdmin auto-creation with broker auto.create.topics.enable
  • Believing you must call AdminClient.createTopics yourself

context

resolving links…

open as a page

How do you configure partitions, replicas, and topic-level settings (like a compacted topic) using TopicBuilder?

level: middleimportance: should knowfreq 35%

basics

~10 s

Use TopicBuilder.name("t").partitions(n).replicas(m), then add topic configs via .config(key, value) or .compact()/.configs(map). .build() returns a NewTopic bean that KafkaAdmin provisions.

open as a page

What happens to KafkaAdmin topic provisioning if the Kafka broker is unavailable when the Spring context starts, and how do you control that behavior?

level: seniorimportance: should knowfreq 22%

basics

~10 s

By default it's non-fatal: KafkaAdmin logs that it couldn't reach the broker and the app still starts, but the declared topics are not created. Set fatalIfBrokerNotAvailable (spring.kafka.admin.fail-fast) to make startup fail instead.

open as a page

At startup, what does KafkaAdmin do when a declared NewTopic already exists on the broker but with different partitions, replication factor, or configs?

level: seniorimportance: should knowfreq 28%

basics

~20 s

It won't recreate the topic. It can increase partitions to match a higher declared count (never decrease). It ignores replication-factor differences. Topic configs are only altered if modifyTopicConfigs is enabled; otherwise differences are left alone.

open as a page

Should you rely on Spring's KafkaAdmin auto-creation for provisioning topics in production? Discuss the trade-offs and how you'd structure it.

level: principalimportance: nice to knowfreq 25%

basics

~20 s

It's great for local/dev and convenience, but as the sole production strategy it's weak: it won't change replication factor, tolerates config drift by default, can silently create 1/1 topics, and ties correct topic layout to app startup. Prefer infra-as-code for prod and use KafkaAdmin declaratively.

open as a page