What is the grace period in Kafka Streams windowing (ofSizeAndGrace), and what happens to records that arrive after it?
answer
- Grace = late-data allowance after window end
- Closes when streamTime > end + grace
- Dropped → dropped-records metric
- KIP-633: default grace 24h → 0, grace mandatory
- retention >= size + grace
basics
~20 sThe grace period is extra time after a window's end during which late (out-of-order) records are still accepted and update the window's result. Once stream time passes window-end + grace, the window is closed and later records are dropped.
solid answer
~40 sGrace period is the allowance for out-of-order/late data. A window covers [start, end); after `end`, Kafka Streams still admits records belonging to that window until **stream time** (the max event-time seen so far) advances past `end + grace`. Records arriving within grace re-trigger the aggregation and emit updated results. Once `streamTime > windowEnd + grace`, the window is **closed**: late records for it are silently dropped (and counted via the `dropped-records` / late-record metrics). You set it with `TimeWindows.ofSizeAndGrace(size, grace)`, `SlidingWindows.ofTimeDifferenceAndGrace(diff, grace)`, `SessionWindows.ofInactivityGapAndGrace(gap, grace)`. Since the older `until()` and the no-grace `ofSize`/`of` factories were deprecated (KIP-633), grace is mandatory and the default changed from 24h to 0. Grace trades latency/completeness against state size: longer grace keeps windows open longer, holding more state and delaying final results, especially relevant when using suppress() with emit-on-window-close.
go deeper
Know grace = extra time to accept late records; after it, late records are dropped.
Tie grace to stream time and the ofSizeAndGrace API; know dropped-records metric exists.
Explain KIP-633 default change, retention >= size+grace constraint, and grace's effect on suppress latency.
Trade off completeness vs latency vs state memory across grace/suppress/retention for SLAs, and design around the burst-close behavior of monotonic stream time.
## The problem grace solves Kafka Streams processes by **event time**, not arrival time. Records can arrive **out of order** (a record with timestamp 10:00:30 may show up after one timestamped 10:01:10, due to producer retries, partition skew, reprocessing, etc.). A window like [10:00, 10:01) needs to know: *how long do I keep accepting late records before I declare the result final?* That allowance is the **grace period**. ## Stream time — the clock that matters **Stream time** is the maximum event-time timestamp Kafka Streams has observed on a task so far. It is **monotonic** (never goes backward) and advances only when a record with a higher timestamp arrives. Window closing is driven by stream time, NOT wall-clock time. So if no new data arrives, stream time doesn't advance and windows don't close. ## Lifecycle of a window 1. Window `[start, end)` is created when the first matching record arrives. 2. While `streamTime <= end + grace`, any record with a timestamp in `[start, end)` is admitted and **re-aggregates** the window, producing an updated (changelog) result. 3. Once `streamTime > end + grace`, the window is **closed**. 4. A record for a closed window is **dropped** — it never updates the result. Drops are observable via metrics: `dropped-records-total` (and historically `late-record-drop`). ## API and KIP-633 Grace is set in the window factory: - `TimeWindows.ofSizeAndGrace(Duration size, Duration grace)` - `SlidingWindows.ofTimeDifferenceAndGrace(Duration diff, Duration grace)` - `SessionWindows.ofInactivityGapAndGrace(Duration gap, Duration grace)` **KIP-633** deprecated the old `until()` method and the grace-less factories (`TimeWindows.of`, `ofSize`) and made grace an explicit, required argument. It also **changed the default grace from 24 hours to 0**, so naive migration can start dropping previously-accepted late data — a classic gotcha. ## Interaction with retention Grace ≤ retention. The window store must physically **retain** a window at least until `end + grace` so late records can update it. Retention (`Materialized.withRetention` / `windowstore retention`) must be **at least** size + grace, or Streams throws at build time. Retention can be longer than grace if downstream Interactive Queries need to read old windows. ## Interaction with suppress() If you want exactly one final result per window (instead of a stream of intermediate updates), you pair grace with `suppress(Suppressed.untilWindowCloses(...))`. The suppression buffer holds results until `streamTime > end + grace`, then emits the final value. A short grace gives low latency but risks dropping late data; a long grace gives completeness but high latency and more buffer/state memory. ## Edge cases - Grace=0 means only in-order or same-timestamp records count; anything strictly later in stream-time is dropped. - Because stream time is per-task and monotonic, a long quiet period followed by one high-timestamp record can suddenly close (and emit/finalize) many windows at once.
- What clock drives window closing — wall-clock or stream time?Stream time: the monotonic maximum event-time timestamp seen on the task. Windows close only when stream time exceeds window-end + grace, so windows won't close if no new (higher-timestamp) data arrives.
- What changed about grace defaults in KIP-633 and why is it a migration hazard?KIP-633 deprecated until()/grace-less factories and changed the default grace from 24h to 0. Migrating code that relied on the implicit 24h grace will suddenly drop late records that were previously accepted.
saying these in an interview costs you the question
- Saying grace is measured against wall-clock time instead of stream time
- Claiming late records past grace are buffered/reprocessed (they are dropped)
- Forgetting that retention must be >= size + grace
- Assuming the default grace is still 24h (it is 0 since KIP-633)