skip to content

What is the broker.rack configuration in Kafka, and why does it matter for a cluster spread across availability zones?

level: juniorimportance: must knowfreq 70%

answer

  1. per-broker AZ label
  2. spread replicas across racks
  3. survive an AZ outage
  4. best-effort, not enforced
  5. no auto-move of existing replicas

basics

~20 s

broker.rack labels each broker with its location (e.g. its availability zone). Kafka uses these labels to spread the copies of each partition across different zones, so losing one zone does not lose all copies of the data.

solid answer

~40 s

broker.rack is a per-broker string config that tags a broker with a failure domain, typically the availability zone (AZ) it runs in, e.g. broker.rack=us-east-1a. When a topic is created or partitions are reassigned, Kafka's rack-aware replica placement tries to put each partition's replicas on brokers in different racks. The goal is that the replicas of any one partition span multiple AZs, so an entire AZ outage does not take all replicas (and thus the partition) offline. It also influences rack-aware consumer/follower fetching when configured. Without broker.rack set, Kafka treats all brokers as one rack and may pile all replicas of a partition into a single AZ, defeating the point of a multi-AZ deployment.

go deeper

for a junior

Know it is a per-broker label (usually the AZ) that makes Kafka spread partition copies across zones to survive a zone outage.

for a middle

Explain best-effort placement, that it must be set per broker, and that it does not retroactively move replicas.

for a senior

Connect it to reassignment tooling, rack count vs replication factor trade-offs, and its role in follower-fetch reads.

for a principal

Reason about it as part of a multi-AZ availability design: rack count, RF, min.insync.replicas, and failure-domain modeling together.

## What broker.rack is Kafka stores each topic partition as multiple **replicas** (copies) on different brokers. The number of copies is the **replication factor**. One replica is the **leader** (handles reads/writes); the others are **followers** that copy the leader's data. A **rack** in Kafka terminology is a *failure domain* — a group of brokers that can fail together. In the cloud, the natural failure domain is an **availability zone (AZ)**: an isolated datacenter within a region. `broker.rack` is a string config set on each broker (in `server.properties`) that names which failure domain that broker belongs to, e.g.: ``` broker.rack=us-east-1a ``` ## Why it matters When Kafka assigns replicas to brokers (at topic creation, partition increase, or reassignment), the **rack-aware replica assignment** algorithm tries to place the replicas of each partition on brokers in *as many distinct racks as possible*. So with replication factor 3 and three AZs, you ideally get one replica in each AZ. The payoff: if an entire AZ goes down, every partition still has surviving replicas in the other AZs, so the cluster keeps serving. Without `broker.rack`, Kafka considers every broker to be in the same (unnamed) rack and may place all 3 replicas of a partition on brokers that happen to be in the same AZ. Then one AZ outage takes that partition completely offline — data is unavailable until the AZ returns. ## Edge cases / details - `broker.rack` is **not** automatically derived; an operator (or the cloud provisioning/Helm chart) must set it correctly per broker. - Rack-awareness is **best effort**: if there are fewer racks than the replication factor, some racks will hold more than one replica. - Changing `broker.rack` on an existing broker does **not** move existing replicas; you must run a partition reassignment to rebalance. - It also feeds **rack-aware fetching** (KIP-392 follower fetching) so consumers can read from a same-AZ replica to cut cross-AZ network cost/latency. - Rack info is exposed in metadata and used by tools like `kafka-reassign-partitions` (with `--enable-rack-aware`, which is on by default when racks are present).

  • If you add broker.rack to brokers that already host topics, are existing partitions rebalanced automatically?
    No. broker.rack only affects new placement decisions. Existing replica assignments are untouched until you run a partition reassignment (e.g. kafka-reassign-partitions), which can take rack info into account.
  • What happens if you have replication factor 3 but only 2 racks defined?
    Rack-awareness is best-effort, so it spreads as widely as it can: two replicas land in distinct racks and the third doubles up in one of them. You no longer survive an arbitrary single-rack loss for every partition without care, so 3 racks for RF=3 is preferred.

saying these in an interview costs you the question

  • Saying broker.rack physically moves data when changed (it only affects future placement)
  • Claiming rack-awareness guarantees one replica per AZ (it is best-effort and depends on rack/RF counts)
  • Confusing broker.rack with consumer rack.id used for follower fetching

context

resolving links…