In Debezium, what do the snapshot-window-open and snapshot-window-close signals do?

## Why a window is needed During an incremental snapshot two producers are emitting events for the same table at the same time: the chunk reader, running ordinary SELECTs against current data, and the log reader, following live changes. Take a naive interleaving. The chunk reader SELECTs row 42 and holds value `A`. A user then updates row 42 to `B`, and the log reader emits that update. The chunk reader finally flushes its buffer and emits row 42 as a read event with value `A`. A sink applying events in order now holds `A` — a value that is older than what it had a moment ago. The snapshot has corrupted the destination. Ordering the two producers is not possible in general, because the SELECT ran at some unknown point relative to the log. What *is* possible is to establish two known points in the log and reason about the interval between them. ## The mechanism For every chunk, Debezium performs three steps. First it writes a row into the signalling table with type `snapshot-window-open`. Second it runs the chunk SELECT — the next N rows in primary-key order after the last chunk's high-water key — into an in-memory buffer keyed by primary key. Third it writes `snapshot-window-close` into the signalling table. Both of those writes are ordinary DML on a captured table, so both appear in the transaction log, and the connector sees them come back through its own streaming path. That gives it a precisely delimited interval. Streamed change events are always emitted as they arrive; but for any event arriving **between** the open and close markers whose key is present in the buffer, the connector deletes that key from the buffer. When the close marker arrives, whatever is left in the buffer is emitted as `r` events. The invariant that falls out is simple and worth stating in exactly these terms: **the log always wins**. If a row changed during the window, the destination gets the live change and never the older snapshot read. If it did not change, the destination gets the snapshot read. Either way it gets one value, and it is the fresher one. ## What this buys operationally Because correctness comes from log positions rather than from holding the source still, no table lock is required and the streaming feed is never paused. The snapshot can run for days against a huge table without the source database noticing anything beyond the read load, and the pipeline stays current throughout. It also means the backfill is interruptible: the connector records the last completed chunk's high-water key in its offsets, so a restart resumes at the next chunk rather than re-reading the table. ## Tuning `incremental.snapshot.chunk.size` sets how many rows one chunk reads. Larger chunks mean fewer window round-trips — fewer signalling writes, less per-chunk overhead — but a bigger buffer held in connector memory and a longer window during which more keys can be invalidated. Smaller chunks are gentler on memory and on the source, at the cost of far more signalling writes. This is a real knob to reach for when a backfill is either too slow or pushing the connector's heap. Recent releases also expose a watermarking strategy choice: the default writes an open and a close row, while an alternative writes and then deletes the marker row so the signalling table does not accumulate them. That matters when the signalling table lives in a database whose owners object to unbounded growth or to extra log volume. ## The read-only variant Writing to the source is not always permitted. The MySQL connector supports a read-only incremental snapshot that establishes the same window boundaries using log positions (GTIDs) instead of writing marker rows, combined with signals delivered over Kafka rather than through a source table. Same algorithm, same guarantee, no writes to the source. ## Failure modes to know If the signalling table is not captured, the marker writes never come back through the streaming path and the windows never close — the symptom is a snapshot that appears to start and then produce nothing. If the connector's memory is undersized relative to the chunk size, large chunks of wide rows can pressure the heap. And if a table has no usable primary key, chunking cannot proceed at all unless a surrogate key is supplied, because the whole scheme depends on being able to identify the same row in the buffer and in the log.