skip to content

What happens to KafkaAdmin topic provisioning if the Kafka broker is unavailable when the Spring context starts, and how do you control that behavior?

level: seniorimportance: should knowfreq 22%

answer

  1. fatalIfBrokerNotAvailable default false
  2. app starts, topics not created, no retry
  3. spring.kafka.admin.fail-fast=true to crash
  4. setAutoCreate(false) + initialize() to defer
  5. don't depend on it for prod topics

basics

~10 s

By default it's non-fatal: KafkaAdmin logs that it couldn't reach the broker and the app still starts, but the declared topics are not created. Set fatalIfBrokerNotAvailable (spring.kafka.admin.fail-fast) to make startup fail instead.

solid answer

~40 s

KafkaAdmin's `fatalIfBrokerNotAvailable` defaults to **false**, so a broker that's unreachable at startup does not stop the application — KafkaAdmin logs the failure and the context comes up, but the declared NewTopic beans are simply not created, and there's no automatic retry loop that provisions them once the broker returns. If you want a hard dependency, set `setFatalIfBrokerNotAvailable(true)` (Boot: `spring.kafka.admin.fail-fast=true`) so the context fails fast when topics can't be created. You can also disable auto-provisioning entirely with `setAutoCreate(false)` and call `kafkaAdmin.initialize()` yourself later (e.g., after the broker is confirmed up). In practice many teams keep it non-fatal but don't rely on it for topic existence — topics are provisioned by infrastructure, and the app just consumes/produces.

code

java · 12 lines
java
// Make a missing broker at startup fail the context (hard Kafka dependency).
// Boot equivalent: spring.kafka.admin.fail-fast=true
@Bean
KafkaAdmin.KafkaAdminCustomizer failFast() {
    return admin -> admin.setFatalIfBrokerNotAvailable(true);
}

// Or defer provisioning entirely and trigger it yourself later.
@Bean
KafkaAdmin.KafkaAdminCustomizer deferProvisioning() {
    return admin -> admin.setAutoCreate(false); // later: kafkaAdmin.initialize();
}

go deeper

for a junior

Know that by default the app still starts even if the broker is down.

for a middle

Know fatalIfBrokerNotAvailable / spring.kafka.admin.fail-fast toggles fatal behavior.

for a senior

Explain there's no auto-retry, the autoCreate/initialize() escape hatch, and the degrade-vs-crash trade-off.

for a principal

Set organization-wide policy on Kafka-as-startup-dependency and decouple prod topic provisioning from app startup.

## Default: broker down at startup is *not* fatal `KafkaAdmin.fatalIfBrokerNotAvailable` defaults to **false**. Consequences when the broker is unreachable during context init: - KafkaAdmin attempts to create topics, the AdminClient call times out/fails, KafkaAdmin **logs an error/warning**, and initialization continues. - The application context **starts successfully**. - The declared topics are **not created**. - There is **no built-in background retry** that provisions the topics later when the broker recovers. (Producers may still create them lazily *if* the broker has `auto.create.topics.enable`, but that's a separate broker feature and yields default partitions/replicas — you lose your declared layout.) ## Making it fatal ```java @Bean KafkaAdmin.KafkaAdminCustomizer failFast() { return admin -> admin.setFatalIfBrokerNotAvailable(true); } ``` Boot property equivalent: `spring.kafka.admin.fail-fast=true`. Now, if topics can't be created at startup, KafkaAdmin throws and the **context fails to start** — turning Kafka into a hard startup dependency. Choose this when the service is useless without its topics and you'd rather crash-loop than run degraded. ## Disabling / deferring auto-creation - `setAutoCreate(false)` stops KafkaAdmin from provisioning during initialization. You then call `kafkaAdmin.initialize()` manually at a moment of your choosing (e.g., a readiness check after the broker is confirmed reachable, or never, if infra owns topics). ## Other relevant KafkaAdmin knobs - `operationTimeout` (`spring.kafka.admin.operation-timeout`) — how long AdminClient operations wait. - `closeTimeout`, `modifyTopicConfigs`, `bootstrapServersSupplier` (dynamic bootstrap servers, e.g. for embedded/test brokers). ## Design guidance - **Fail-fast vs degrade:** fail-fast surfaces misconfiguration early but couples startup to Kafka availability. Non-fatal keeps the app up but risks running against missing topics. Pick deliberately per service. - **Don't treat app-side auto-creation as your production provisioning strategy.** Relying on it means correct topic layout depends on the broker being up at the right moment and on nobody's producer lazily creating a mis-shaped topic first. Provision topics via infrastructure/GitOps and let KafkaAdmin be a convenience for local/dev.

  • The broker was down at startup, comes up 30 seconds later — are the declared topics created automatically?
    No. KafkaAdmin provisions once during initialization and has no retry loop; if that failed, the topics stay uncreated until the next restart (or a manual initialize()). They might appear only if broker auto.create.topics.enable creates them lazily on first use, with default layout.
  • When would you deliberately choose fail-fast=true?
    When the service cannot function without its topics and you prefer an obvious startup failure / crash-loop (caught by health checks and deploy gates) over a silently degraded instance producing/consuming against missing topics.

saying these in an interview costs you the question

  • Assuming a missing broker always fails context startup (default is non-fatal)
  • Believing KafkaAdmin retries provisioning after the broker recovers
  • Confusing fail-fast for topics with consumer/producer connection retries

context

resolving links…