skip to content

How does the Event Hubs Throughput-Unit / Processing-Unit capacity model differ from sizing an Apache Kafka cluster, and how does that interact with partitions?

level: middleimportance: should knowfreq 40%

answer

  1. TU/PU/CU = capacity, brokers ≠
  2. 1 TU ≈ 1 MB/s / 1000 ev/s in
  3. partitions fixed at creation (Standard)
  4. Auto-Inflate scales up only
  5. TUs namespace-wide → noisy-neighbor throttle

basics

~20 s

On Apache Kafka you size brokers, disks, and partitions. On Event Hubs you buy Throughput Units (or Processing/Capacity Units) that cap MB/s and events/s regardless of partition count; partitions are fixed at creation and mainly affect parallelism, not capacity.

solid answer

~50 s

In self-managed Kafka, throughput is a function of broker count, CPU, disk, network, and how partitions spread load — you scale by adding brokers and rebalancing partitions. Event Hubs decouples capacity from topology: you provision **Throughput Units (TU)** on Standard (each ~1 MB/s or 1000 events/s in, 2 MB/s out), **Processing Units (PU)** on Premium, or **Capacity Units (CU)** on Dedicated. Exceeding your units yields Kafka-style throttling responses. Partitions still exist (parallelism/ordering unit for consumers), but on Standard they are chosen at hub creation and historically immutable, and they do NOT raise your throughput ceiling — that's set by the units you bought. So the mental model flips: in Kafka partitions ≈ scalability lever; in Event Hubs units ≈ scalability lever and partitions ≈ consumer-parallelism lever. This affects auto-scaling, cost, and how you plan consumer groups.

go deeper

for a junior

Know capacity is sold as Throughput Units, not brokers.

for a middle

Explain the TU/PU numbers and that partitions are fixed-at-creation parallelism, not capacity.

for a senior

Reason about Auto-Inflate, namespace-wide throttling, and partition-count caps on consumer scale-out.

for a principal

Weigh the unit/partition model against Kafka sizing for migration cost, elasticity, and lock-in.

## Background: how Kafka capacity works In Apache Kafka, a topic is divided into **partitions**, each an ordered append-only log hosted on a broker. Throughput scales because partitions spread reads/writes across many brokers and disks. To handle more load you typically: add brokers, increase partition count (so more parallel producers/consumers and more spread), and tune `replication.factor`, batching, and disk. **Partitions are the primary scalability and parallelism unit**, and you can increase a topic's partition count later (though that breaks key→partition stability). ## Event Hubs capacity model Event Hubs sells *capacity*, not infrastructure: - **Standard tier → Throughput Units (TU):** each TU ≈ 1 MB/s **or** 1000 events/s ingress, 2 MB/s **or** 4096 events/s egress. You can buy a small number of TUs and enable **Auto-Inflate** to raise them automatically up to a cap (it does not auto-deflate). - **Premium tier → Processing Units (PU):** isolated CPU/memory allocations; more predictable, supports larger retention and bigger messages. - **Dedicated tier → Capacity Units (CU):** a whole reserved cluster. The throughput ceiling is set by the **units**, applied at the **namespace** level (Standard) — it is shared across all event hubs in that namespace. When you exceed it you get throttled (the Kafka client sees throttle/`RetriableException` style backpressure). ## Partitions in Event Hubs Partitions still exist and still define **ordering** (within a partition) and **consumer parallelism** (max useful concurrent readers per consumer group ≈ partition count). BUT: - On Standard, partition count is **chosen at event-hub creation** and was historically **immutable** (you couldn't grow it without recreating the hub). Newer capabilities allow partition increase on some tiers, but the classic gotcha is plan it up front. - More partitions do **not** raise your throughput cap — that's the TU/PU's job. ## Why the distinction matters 1. **Sizing:** In Kafka you co-design brokers + partitions for throughput. In Event Hubs you size units for throughput and partitions only for parallelism/ordering. Over-provisioning partitions in Event Hubs buys parallelism but no capacity. 2. **Auto-scaling:** Kafka scaling = broker/partition operations. Event Hubs scaling = increase TUs (Auto-Inflate) — operationally trivial but only up to tier limits, and partitions can't easily follow. 3. **Cost model & lock-in:** TUs/PUs are an Azure-specific commercial abstraction. Designing around them (and around fixed partition counts) is one of the migration frictions when moving on/off Event Hubs. ## Edge cases - Auto-Inflate scales **up only**, never down — cost can creep. - Because TUs are namespace-wide on Standard, one noisy event hub can throttle siblings. - A consumer group can't usefully exceed partition count in parallel readers, so an under-partitioned hub caps consumer scale-out even with plenty of TUs.

  • If you add more partitions to an Event Hub, does your maximum ingest throughput go up?
    No. Throughput is capped by Throughput/Processing Units. More partitions add ordering streams and consumer parallelism, not capacity.
  • What is a limitation of Auto-Inflate?
    It only scales TUs upward (up to a configured max) and never scales them back down, so cost can ratchet up after a spike.

saying these in an interview costs you the question

  • Claiming you add brokers in Event Hubs to scale — there are no user-visible brokers; you add units.
  • Saying more partitions increase throughput on Event Hubs (capacity is unit-bound).
  • Forgetting that Standard TUs are namespace-wide and can cause noisy-neighbor throttling.

context

resolving links…