skip to content

How do you set a produce byte-rate quota for a specific user using kafka-configs, and how do default quotas interact with specific ones?

level: middleimportance: must knowfreq 55%

answer

  1. kafka-configs --alter --add-config
  2. --entity-type users/clients, --entity-default
  3. most-specific-wins resolution order
  4. bytes not bits; per broker
  5. --describe / --delete-config, no restart

basics

~10 s

Use kafka-configs.sh --alter --add-config 'producer_byte_rate=...' against --entity-type users --entity-name <user>. A specific entity quota overrides the cluster --entity-default for that level.

solid answer

~40 s

You alter quotas as dynamic configs with kafka-configs.sh: `kafka-configs.sh --bootstrap-server b:9092 --alter --add-config 'producer_byte_rate=1048576,consumer_byte_rate=2097152' --entity-type users --entity-name alice`. The same command supports `--entity-type clients` (client-id) and combining `--entity-type users --entity-type clients` for a (user, client-id) pair. Replacing `--entity-name X` with `--entity-default` sets the cluster default at that level. Resolution is most-specific-wins: the broker checks (user, client-id), then user, then client-id, then the matching defaults. So a per-user quota overrides the users-default; a (user,client-id) quota overrides the bare user quota. Remove a quota with `--delete-config producer_byte_rate`. Inspect with `--describe`. All of this is runtime and needs no restart, and on a SASL cluster the 'user' is the authenticated principal.

go deeper

for a junior

Recognize that kafka-configs --alter sets a quota and that defaults exist; details optional.

for a middle

Write the exact CLI for a user quota and explain specific-overrides-default resolution.

for a senior

Reason about the full resolution order, per-broker scaling, and the auth dependency for per-user enforcement.

for a principal

Design a quota policy: sensible defaults plus carve-outs, tie principals to SASL identity, and document per-broker capacity math for tenants.

## The tool `kafka-configs.sh` (the `kafka-configs` CLI) manages **dynamic configs**, including quotas. Quotas are not in `server.properties`; they live in cluster metadata and change at runtime. ## Setting a quota ``` kafka-configs.sh --bootstrap-server broker:9092 \ --alter --add-config 'producer_byte_rate=1048576,consumer_byte_rate=2097152,request_percentage=200' \ --entity-type users --entity-name alice ``` - `producer_byte_rate=1048576` → 1 MiB/s produce cap **per broker** for user `alice`. - `consumer_byte_rate` → fetch cap. `request_percentage=200` → up to 2 threads' worth of broker IO/network time. - `--entity-type users --entity-name alice`: the principal `alice` (as authenticated by SASL/SSL). ## Entity types and combinations - **User only:** `--entity-type users --entity-name alice`. - **Client-id only:** `--entity-type clients --entity-name svc-orders`. - **(user, client-id) pair:** `--entity-type users --entity-name alice --entity-type clients --entity-name svc-orders`. ## Defaults vs specific — the resolution order Replace `--entity-name` with `--entity-default` to set a cluster-wide default at that level: ``` kafka-configs.sh ... --alter --add-config 'producer_byte_rate=512000' \ --entity-type users --entity-default ``` The broker resolves the quota for an incoming request by **most-specific match wins**, in this order: 1. `(user, client-id)` specific 2. `(user, client-id-default)` 3. `user` specific 4. `user-default` 5. `client-id` specific 6. `client-id-default` So a quota set on `alice` overrides the users-default; a `(alice, svc-orders)` quota overrides the bare `alice` quota. This lets you set a broad default for all users and carve out exceptions for heavy or throttled tenants. ## Inspecting and removing - Describe: `kafka-configs.sh ... --describe --entity-type users --entity-name alice`. - Remove one key: `--alter --delete-config producer_byte_rate --entity-type users --entity-name alice`. ## Edge cases / gotchas - The values are **bytes**, not bits — 1048576 = 1 MiB/s, a common off-by-8 mistake. - Quotas are **per broker**: cluster throughput for the client is (per-broker quota) × (brokers hosting its leader partitions). - On an unauthenticated (PLAINTEXT) cluster every connection maps to the same default user `ANONYMOUS`, so per-user quotas are meaningless without SASL; client-id quotas still work but are trivially spoofable. - Changing a quota takes effect almost immediately on all brokers; in-flight throttled clients recompute against the new limit.

  • If you set a users-default of 1 MB/s and a specific quota of 5 MB/s on user 'alice', what does alice get?
    5 MB/s. The specific user quota is more specific than the users-default and wins. Other users without a specific quota fall back to the 1 MB/s default.
  • Why might per-user quotas be ineffective on a PLAINTEXT listener?
    Without authentication every client maps to the principal ANONYMOUS, so all clients share one user entity. You need SASL/mTLS to get distinct principals; otherwise rely on client-id quotas, which clients can freely spoof.

saying these in an interview costs you the question

  • Putting quotas in server.properties and requiring a restart — they are dynamic configs.
  • Treating the byte-rate value as bits or as a cluster-wide total.
  • Believing a users-default overrides a specific user quota — specific always wins.
  • Assuming per-user quotas work without authentication (they collapse to ANONYMOUS).

context

resolving links…