What is a GlobalKTable, and how does it differ from a KTable in partitioning, joins, and co-partitioning requirements?

## The problem GlobalKTable solves When you join two streams/tables in Kafka Streams, the runtime must be able to find the matching record **locally**, because each stream task only owns a subset of partitions. ### KTable: partitioned (sharded) state A `KTable` source topic with N partitions becomes N stream tasks; **task i holds only the keys whose records landed in partition i**. To join a `KStream` with a `KTable`, both sides must be **co-partitioned**: 1. **Same key** on both sides, 2. **Same number of partitions**, and 3. **Same partitioning strategy** (same partitioner / key serialization), so a given key lands in the same partition number on both topics. If these don't hold, the matching record may live on a different instance and the join silently misses rows. Kafka Streams enforces a partition-count check and will throw `TopologyException` for KTable-KTable joins that aren't co-partitioned; for keyed operations it inserts an automatic **repartition topic** when you change the key. ### GlobalKTable: fully replicated state `StreamsBuilder.globalTable(topic)` builds a `GlobalKTable`. **Every application instance consumes ALL partitions** of the source topic into its own local store. Consequences: - **No co-partitioning required** — the full dataset is local everywhere. - You join with a **`KeyValueMapper<K, V, GK>`** that extracts the join key from the *stream* record, so you can join on a **foreign key / non-primary field**, e.g. order.customerId → customer table keyed by customerId, even if the order stream is keyed by orderId. - Loaded by a dedicated **GlobalStreamThread**, populated on startup (bootstrap) before processing begins. ### Join-time semantics (a subtle, senior-level point) - **KStream-KTable** joins are **timestamp-synchronized**: the runtime aligns the two sides by event time using `max.task.idle.ms` buffering, so the KTable value used is the one current as of the stream record's timestamp. - **KStream-GlobalKTable** joins are **NOT** timestamp-synchronized: the GlobalKTable is updated **eagerly and independently** of stream time. A lookup uses whatever the global store currently holds. This is usually fine for slowly-changing reference data but can produce subtle non-determinism if the global data changes rapidly. ### Trade-offs / when to use - Use **GlobalKTable** for **small, relatively static reference/lookup data** that every instance needs (country codes, FX rates, feature flags) and when you need a **foreign-key-style** join without repartitioning the (often large) stream. - Avoid it for **large, fast-changing** datasets: full replication multiplies storage/memory by instance count and the bootstrap load grows. - Prefer **KTable** when the data is large and naturally co-partitioned with the stream, or when you need timestamp-aligned, event-time-correct joins. ### Edge cases - A GlobalKTable does NOT create a repartition topic for the join key on the stream side — the mapper just extracts a key for the lookup; no reshuffling of the stream happens. - Only **inner** and **left** joins are supported for KStream-GlobalKTable (no outer); KTable-KTable supports inner/left/outer. - GlobalKTable does not participate in the normal task/partition assignment, so it isn't repartitioned and isn't part of a sub-topology boundary the way a repartition is.