skip to content

Broker and Topic Configuration Management

Static server.properties versus dynamic configs, topic-level overrides, and which value actually takes effect. Interviewers ask because config precedence explains many 'I changed it and nothing happened' incidents.

part ofApache Kafkaoverview, primer and where to startread it →
on this pageshow

questions

6

How do you use kafka-configs.sh to alter a topic-level config, and what does the command look like?

level: juniorimportance: must knowfreq 65%

answer

  1. --entity-type topics --entity-name <t>
  2. --alter --add-config k=v,k=v
  3. --delete-config reverts to default
  4. --describe (--all for sources)
  5. --bootstrap-server, never --zookeeper

basics

~10 s

Use kafka-configs.sh with --bootstrap-server, --entity-type topics, --entity-name <topic>, and --alter --add-config key=value. To remove an override use --delete-config key. --describe shows current overrides.

solid answer

~40 s

kafka-configs.sh is the CLI for reading and changing entity-level configs. You pass --bootstrap-server, then identify the entity with --entity-type (topics, brokers, users, clients, ips) and --entity-name (the topic name, broker id, etc.). --alter --add-config 'retention.ms=604800000' sets or overrides one or more comma-separated key=value pairs; --delete-config 'retention.ms' removes the override so the topic falls back to the cluster default. --describe lists the configs explicitly set on that entity. For example: kafka-configs.sh --bootstrap-server localhost:9092 --entity-type topics --entity-name orders --alter --add-config retention.ms=86400000,cleanup.policy=compact. Topic configs set this way are stored per topic and override the matching broker-level default (e.g. log.retention.ms). The same tool sets dynamic broker configs with --entity-type brokers and either --entity-name <id> (per-broker) or --entity-default (cluster-wide).

go deeper

for a junior

Recall the flag shape: --entity-type/--entity-name plus --alter --add-config k=v.

for a middle

Know --delete-config to revert, and --describe --all to see effective sources.

for a senior

Map topic configs to their broker log.* counterparts and reason about override scope.

for a principal

Standardize config-change tooling/automation (AdminClient over scripts) and guardrails for production alters.

## What kafka-configs.sh does `kafka-configs.sh` (just `kafka-configs` on some packages) is the command-line tool for **describing** and **altering** configuration on Kafka *entities*. An entity is the thing the config attaches to. The common entity types are: - `topics` — per-topic overrides - `brokers` — dynamic broker configs - `users`, `clients`, `ips` — quotas and connection limits ## Anatomy of an --alter command ``` kafka-configs.sh \ --bootstrap-server localhost:9092 \ --entity-type topics \ --entity-name orders \ --alter \ --add-config retention.ms=86400000,max.message.bytes=2097152 ``` - `--bootstrap-server` — any broker address; the tool talks to the cluster via the AdminClient protocol. (The old `--zookeeper` flag is removed in modern Kafka — do not use it.) - `--entity-type topics` + `--entity-name orders` — identify the topic. - `--alter` — we are changing config. - `--add-config` — comma-separated `key=value` pairs to set/override. Setting a key that already has an override just replaces it. ## Removing an override ``` kafka-configs.sh --bootstrap-server localhost:9092 \ --entity-type topics --entity-name orders \ --alter --delete-config retention.ms ``` This removes the per-topic `retention.ms`, so the topic reverts to the broker default (`log.retention.ms`). ## Describing ``` kafka-configs.sh --bootstrap-server localhost:9092 \ --entity-type topics --entity-name orders --describe ``` By default this lists only configs **explicitly set** on the topic (dynamic overrides). Add `--all` to also show the inherited defaults and where each value comes from (its config source). ## Topic config vs broker config naming Many topic-level configs have a broker-level counterpart with a `log.` prefix: topic `retention.ms` overrides broker `log.retention.ms`; topic `cleanup.policy` overrides `log.cleanup.policy`. Setting it at the topic level affects only that topic. ## Common gotchas - Values are in the config's native unit (milliseconds, bytes) — `retention.ms=86400000` is one day, not seconds. - `--add-config` is additive/replacing, not a full replace of all configs; other overrides stay untouched. - Quoting matters in shells when a value contains commas or special characters; wrap the whole `key=value,key=value` string in quotes.

  • How do you make a topic config revert to the cluster default after you set an override?
    Run --alter --delete-config <key> for that topic; it removes the override and the topic inherits the broker-level default again.
  • Why is --zookeeper no longer the right flag?
    Modern Kafka uses the AdminClient protocol via --bootstrap-server; ZooKeeper-based admin access was deprecated and removed, especially under KRaft where there is no ZooKeeper.

saying these in an interview costs you the question

  • Using --zookeeper instead of --bootstrap-server on a modern/KRaft cluster.
  • Thinking --add-config replaces all configs — it only sets/replaces the listed keys.
  • Confusing units (e.g. treating retention.ms as seconds).

context

open as a page

What is the difference between static (server.properties) configs and dynamic configs in Kafka, and why does it matter operationally?

level: juniorimportance: must knowfreq 70%

basics

~10 s

Static configs live in server.properties and only take effect after a broker restart. Dynamic configs are set with kafka-configs at runtime and apply without restarting the broker, so you can tune a live cluster.

open as a page

How do you inspect the effective configuration of a topic or broker, and what is the difference between --describe and --describe --all?

level: middleimportance: should knowfreq 45%

basics

~10 s

Use kafka-configs --describe with --entity-type and --entity-name. By default it shows only explicitly-set overrides. Adding --all also lists inherited defaults and each value's source, giving the full effective config.

open as a page

When setting a dynamic broker config, what is the difference between --entity-name <id> and --entity-default, and when would you use each?

level: middleimportance: should knowfreq 40%

basics

~10 s

--entity-name <brokerId> sets a per-broker config that applies to just that broker. --entity-default sets a cluster-wide default applied to all brokers. Per-broker overrides the cluster default for that broker.

open as a page

Explain the configuration precedence hierarchy in Kafka: how is the effective value of a broker/topic config resolved across per-broker, cluster-default, static, and built-in defaults?

level: seniorimportance: should knowfreq 50%

basics

~10 s

Kafka resolves the most specific source first: per-broker dynamic, then cluster-wide dynamic default, then the static server.properties value, then the hardcoded default. Topic-level overrides win over the broker default for that topic.

open as a page

How does Kafka protect sensitive dynamic broker configs (like SSL keystore passwords), and what is the role of the password encoder secret?

level: seniorimportance: should knowfreq 35%

basics

~10 s

Sensitive dynamic configs (e.g. passwords) are stored encrypted in the metadata store, not in plaintext. Each broker needs password.encoder.secret in server.properties to encrypt/decrypt them, and kafka-configs --describe shows such values as redacted/[hidden].

open as a page