Describe Pulsar's native multi-tenancy and geo-replication model, and contrast it with how Kafka achieves the same goals.
answer
- tenant → namespace → topic
- namespace = policy/quota/auth/replication unit
- geo-repl is a namespace property, async, origin-tagged
- Kafka flat namespace + ACLs + quotas
- Kafka cross-region = MirrorMaker 2 / Cluster Linking
basics
~20 sPulsar bakes in a tenant/namespace/topic hierarchy with per-tenant quotas, auth, and isolation, and supports cross-region replication by configuring clusters on a namespace. Kafka has a flat topic namespace and relies on external tools (ACLs, MirrorMaker 2) to approximate both.
solid answer
~40 sPulsar's namespace model is hierarchical: **tenant → namespace → topic**. A tenant maps to a team/customer; a namespace is an admin/policy unit inside a tenant carrying quotas, retention, backlog limits, auth/ACLs, and which clusters it replicates to. This makes one Pulsar cluster a genuine multi-tenant platform out of the box. **Geo-replication** is a namespace property: list the clusters a namespace spans and Pulsar asynchronously replicates published messages to brokers in those regions (with per-message origin tracking to avoid loops). Kafka, by contrast, has a flat topic namespace with no tenant concept — you simulate tenancy with naming conventions, per-topic ACLs, and quotas. Cross-region copying uses external tooling: **MirrorMaker 2** (Connect-based) or commercial Cluster Linking, which run as separate processes and don't give the seamless single-logical-topic-across-regions that Pulsar's built-in replication does.
go deeper
Know that Pulsar has tenants/namespaces built in and Kafka uses flat topics plus ACLs.
Explain that namespace carries quotas/auth/retention and that Kafka uses MirrorMaker 2 for cross-region copying.
Detail async replication, origin-based loop prevention, replicated subscriptions, and the logical-vs-copied-topic distinction.
Evaluate active-active topologies, RPO on failover, tenant isolation guarantees, and whether built-in tenancy/geo-repl reduces platform sprawl enough to justify Pulsar.
**Multi-tenancy.** *Pulsar* has a built-in three-level naming hierarchy. A fully-qualified topic looks like `persistent://<tenant>/<namespace>/<topic>`: - **Tenant** — top-level isolation unit, typically a team, business unit, or customer. Has its own admin roles and an *allowed clusters* list. - **Namespace** — the policy/administrative grouping within a tenant. *This is where most knobs live:* retention policy, backlog quota, message TTL, deduplication, schema-compatibility rules, authorization (which roles can produce/consume), throttling, and the set of clusters for geo-replication. - **Topic** — the actual stream/queue, persistent or non-persistent. Because quotas, auth, and isolation are first-class at tenant/namespace level, a single Pulsar cluster can safely host many teams/customers — that's what 'native multi-tenancy' means. Pulsar can also pin tenants/namespaces to specific brokers (*broker isolation policies*) so a noisy tenant doesn't starve others. *Kafka* has a **flat** topic namespace — just topic names, no tenant/namespace objects. Multi-tenancy is *convention + add-ons*: prefix topics by team (`teamA.orders`), apply per-principal **ACLs**, set **client quotas** (byte-rate/request-rate per user/client-id), and possibly run separate clusters per tenant for hard isolation. It works, but tenancy is assembled by the operator rather than provided by the platform. **Geo-replication.** *Pulsar* treats replication as a *namespace property*. You define multiple named clusters (each its own brokers + BookKeeper, sharing a global metadata view via *configuration store*). Setting a namespace's replication clusters to, say, `[us-east, eu-west]` makes brokers asynchronously forward locally-published messages to the other regions' brokers. Key mechanics: - It is **async** by default (low local latency; cross-region lag). - Each message carries its **origin cluster**, so replication doesn't loop in active-active topologies — a message originating in eu-west isn't bounced back to eu-west. - Subscriptions can optionally be replicated too (replicated subscriptions) so consumers can fail over regions with cursor continuity. - It supports active-active (every region produces and consumes) without external processes. *Kafka* has no built-in cross-cluster replication. You use: - **MirrorMaker 2 (MM2)** — built on Kafka Connect; it copies topics, consumer-group offsets, and ACLs between clusters, renaming topics with a cluster prefix (e.g. `us-east.orders`) to prevent loops. It's a separate deployment to run, monitor, and tune. - **Cluster Linking** (Confluent, commercial) — byte-for-byte mirroring keeping the same topic name, more seamless but proprietary. The philosophical difference: in Pulsar a topic in a replicated namespace is *one logical topic that exists in several regions*; in Kafka (MM2) you have *distinct topics per cluster* tied together by a copying process and naming convention. **Edge cases / gotchas.** - Pulsar geo-replication is async, so a region failover can lose the not-yet-replicated tail — RPO is non-zero, same as MM2. - Active-active with the same key produced in two regions can still create conflicting orderings; Pulsar's loop-prevention handles duplication, not application-level conflict resolution. - Multi-tenancy isolation in Pulsar is logical by default; a truly hostile/noisy tenant may still need broker isolation policies or separate bookie pools for hardware-level isolation, analogous to Kafka's 'just run a separate cluster' answer.
- How does Pulsar prevent infinite replication loops in an active-active geo-replication setup?Each message records its origin cluster. When a broker would replicate a message, it skips sending it back to the cluster it originated from, so messages don't bounce endlessly between regions.
- What does MirrorMaker 2 do that mimics Pulsar's namespace geo-replication, and where does it fall short?MM2 (Kafka Connect-based) copies topics, consumer offsets, and ACLs across clusters, prefixing topic names to avoid loops. It falls short because it's a separate system to run/tune and produces distinct per-cluster topics rather than one logical topic spanning regions.
saying these in an interview costs you the question
- Saying Kafka has a built-in tenant/namespace hierarchy
- Claiming Pulsar geo-replication is synchronous by default
- Saying MirrorMaker 2 keeps the same topic name and zero ops (it prefixes and is a separate deployment)
- Asserting Pulsar's multi-tenancy gives hardware isolation by default