skip to content

Walk through the describe/alter-config workflow with kafka-configs.sh and AdminClient, including incrementalAlterConfigs versus the legacy alterConfigs.

level: seniorimportance: should knowfreq 45%

answer

  1. describe shows source tag
  2. incrementalAlterConfigs: SET/DELETE/APPEND/SUBTRACT
  3. legacy alterConfigs replaced whole set (footgun)
  4. APPEND/SUBTRACT only for list configs
  5. ConfigResource.Type TOPIC/BROKER/BROKER_LOGGER

basics

~10 s

Use kafka-configs.sh --describe to read a topic/broker's configs and their source, and --alter --add-config/--delete-config to change them. Programmatically, prefer AdminClient.incrementalAlterConfigs (SET/DELETE/APPEND ops) over the deprecated alterConfigs, which replaced the whole config set.

solid answer

~40 s

The workflow: first describe to see current values and their source (DYNAMIC_TOPIC_CONFIG, DEFAULT_CONFIG, STATIC_BROKER_CONFIG, etc.), then alter. CLI: kafka-configs.sh --describe --entity-type topics --entity-name T, and --alter --add-config k=v or --delete-config k. Programmatically, AdminClient exposes describeConfigs(Collection<ConfigResource>) returning the full Config with ConfigEntry sources. For changes, incrementalAlterConfigs takes a map of ConfigResource to a list of AlterConfigOp, each pairing a ConfigEntry with an op type SET, DELETE, APPEND, or SUBTRACT — so you mutate only the keys you name. The legacy alterConfigs is deprecated because it required sending the ENTIRE desired config; any key you omitted was reset to default, a classic footgun. incrementalAlterConfigs is atomic per resource and only touches named keys. ConfigResource.Type can be TOPIC, BROKER, or BROKER_LOGGER.

go deeper

for a junior

Know --describe reads and --alter --add-config/--delete-config changes topic configs.

for a middle

Explain the SET/DELETE ops and reading the source of a config value.

for a senior

Contrast incrementalAlterConfigs vs the clobbering legacy alterConfigs and use APPEND/SUBTRACT correctly.

for a principal

Design safe config-change tooling/automation that uses incremental ops, audits sources, and avoids destructive full replaces.

## The describe step Before changing anything, read current state. The CLI: ``` kafka-configs.sh --bootstrap-server localhost:9092 --describe \ --entity-type topics --entity-name orders ``` This prints each config with a **source** tag explaining where the value comes from: - `DYNAMIC_TOPIC_CONFIG` — an explicit per-topic override. - `DYNAMIC_BROKER_CONFIG` / `DYNAMIC_DEFAULT_BROKER_CONFIG` — dynamic broker-level values. - `STATIC_BROKER_CONFIG` — from server.properties. - `DEFAULT_CONFIG` — the built-in default. Knowing the source tells you whether a value is overridden or inherited, which drives whether you SET or DELETE. Programmatically: ```java ConfigResource cr = new ConfigResource(ConfigResource.Type.TOPIC, "orders"); Map<ConfigResource, Config> cfg = admin.describeConfigs(List.of(cr)).all().get(); cfg.get(cr).entries().forEach(e -> System.out.println(e.name() + "=" + e.value() + " src=" + e.source())); ``` ## The alter step — two APIs ### Legacy alterConfigs (deprecated) `alterConfigs(Map<ConfigResource, Config>)` replaced the **entire** dynamic config set for the resource with what you sent. If you wanted to change one key, you had to first read all configs and resend them; **any key you forgot was reset to its default**. This is the well-known footgun. ### incrementalAlterConfigs (preferred) ```java List<AlterConfigOp> ops = List.of( new AlterConfigOp(new ConfigEntry("retention.ms", "604800000"), AlterConfigOp.OpType.SET), new AlterConfigOp(new ConfigEntry("cleanup.policy", null), AlterConfigOp.OpType.DELETE) ); admin.incrementalAlterConfigs(Map.of(cr, ops)).all().get(); ``` Op types: - **SET** — set/override a key. - **DELETE** — remove the override so it reverts to inherited/default. - **APPEND** / **SUBTRACT** — for **list-valued** configs only (e.g. `cleanup.policy` can be a list, or follower throttle replica lists), add/remove list elements without rewriting the whole list. It only touches the keys you name; everything else is untouched. It is applied atomically per ConfigResource. ## CLI equivalent ``` kafka-configs.sh --alter --entity-type topics --entity-name orders \ --add-config retention.ms=604800000 \ --delete-config cleanup.policy ``` The CLI uses incrementalAlterConfigs under the hood in modern Kafka. ## ConfigResource types - `TOPIC` — per-topic configs. - `BROKER` — dynamic broker configs (some can be changed without restart). - `BROKER_LOGGER` — log4j levels, changeable at runtime. ## Edge cases - Not every broker config is dynamically alterable; read-only ones require a restart and editing server.properties. - incrementalAlterConfigs with DELETE on a key that wasn't overridden is a no-op. - Sensitive configs (passwords) show as null in describe output.

  • Why is the old alterConfigs API dangerous?
    It replaced the entire dynamic config set for the resource, so any key you omitted from the request was reset to its default. incrementalAlterConfigs only touches the keys you name.
  • When would you use the APPEND op type?
    Only for list-valued configs (e.g. adding a replica to a throttle list or a value to cleanup.policy) so you can add an element without rewriting and risking clobbering the existing list.
  • How can you tell whether a topic config value is overridden or inherited?
    describeConfigs returns each ConfigEntry's source — DYNAMIC_TOPIC_CONFIG means overridden; DEFAULT_CONFIG or STATIC_BROKER_CONFIG means inherited.

saying these in an interview costs you the question

  • Recommending alterConfigs without noting it clobbers unspecified keys
  • Saying APPEND/SUBTRACT work on any config (they're list-only)
  • Forgetting that DELETE reverts a key to inherited/default
  • Thinking all broker configs can be changed dynamically without a restart

context

resolving links…