Apache Flink

Both emit a result for the window. FIRE leaves the window's buffered elements in place, so a later firing recomputes over them again; FIRE_AND_PURGE clears the contents afterwards, so the next firing starts from empty. Neither removes the window itself.

Reach for process() when the logic needs keyed state, timers, the record's event-time timestamp, or side outputs. flatMap() only transforms one record into zero or more records with no memory of what came before and no way to act at a future time.

Walk the Flink job graph downstream and find the first task that is busy but not backpressured — that is the bottleneck. Everything upstream of it shows backpressure, everything downstream shows idle time.

Stop the Flink job with a savepoint, then resubmit from that savepoint with a new -p value. The hard ceiling is the job's maximum parallelism, which fixes the number of key groups and cannot be changed without rewriting the state.

Watermarks are generated per input and combined by taking the minimum, so a silent partition's watermark never advances and pins the whole operator's event time. Configure withIdleness on the WatermarkStrategy so idle inputs are excluded from that minimum.

allowedLateness keeps a window's state alive past its watermark deadline, so a straggler triggers an extra firing with an updated result. sideOutputLateData routes records too late even for that grace period to a separate stream instead of dropping them silently.

Mini-batch (table.exec.mini-batch.enabled plus allow-latency and size) buffers rows so each key's state is touched once per batch. On top of it, table.optimizer.agg-phase-strategy = 'TWO_PHASE' pre-aggregates before the shuffle, so the hot key's task receives accumulators instead of raw rows.

Take a savepoint, add the field to the POJO, and restore from that savepoint. Flink evolves POJO and Avro state schemas automatically: added fields get their Java default value, removed fields are dropped, but declared field types and the class name cannot change.

Attach a StateTtlConfig to the state descriptors so entries expire, and back it with explicit cleanup: a KeyedProcessFunction timer that calls state.clear() when a key goes idle. Flink never garbage-collects a key just because it stopped appearing.

Several result rows with different upsert keys can land on one sink key, so the stored value depends on arrival order. Flink 2.3 fails planning unless you fix the key mismatch or add ON CONFLICT DO ERROR, DO NOTHING or DO DEDUPLICATE.

A 24-hour window sliding every minute keeps 1440 windows open per key at once, and every record is assigned to all of them. State is multiplied roughly 1440-fold per key, so checkpoints balloon and the job runs out of memory or disk.

When the sink is naturally idempotent — keyed upserts, partition overwrites — replayed records overwrite rather than duplicate, so at-least-once gives the same final result without alignment stalls, transaction machinery, or output that is invisible until the next checkpoint completes.

Maximum parallelism fixes the number of key groups, the atomic unit by which Flink redistributes keyed state. Changing it changes the key-to-key-group mapping, so existing state can no longer be assigned and must be discarded. Set it once, generously, and never touch it.

Java erases the generic parameter of the Collector in flatMap's signature, so Flink cannot infer the output type from the lambda. Supply it explicitly with .returns(...) or implement FlatMapFunction as a class.

Technically yes, via Queryable State, but it should not: Flink 2.3 ships it only as a deprecated, off-by-default leftover slated for removal. Publish readable state to an external store, or read a savepoint offline with the State Processor API.

An evictor inspects and removes individual elements after the trigger fires, so every element must still exist when the window is evaluated. That forces Flink to buffer the whole window instead of folding records into a single accumulator on arrival.

Under backpressure a barrier crawls through full network queues and alignment stalls the snapshot. Unaligned checkpoints let the barrier overtake queued records and persist the in-flight buffers as part of the snapshot, so checkpoint duration stops depending on throughput.

It stops one source split from racing far ahead of the others in event time. Flink pauses reading from splits whose watermark exceeds the group's minimum by more than the configured drift, bounding the state that accumulates while slow splits catch up.

Reactive mode suits a single standalone application-mode job with modest state and a strong daily traffic swing, where an external controller adds and removes TaskManagers. Steady jobs or very large state are better served by sizing for peak and rescaling rarely.