Explain how the Postgres Debezium connector consumes the WAL: logical replication, output plugins (pgoutput/wal2json), replication slots, and the operational risks of slots.

**WAL and logical decoding.** PostgreSQL's **Write-Ahead Log (WAL)** records every change at the *physical* level (block/page changes) for crash recovery and physical replication. That format is unusable for CDC directly. **Logical decoding** is Postgres's mechanism to translate WAL into a *logical* stream of row-level changes (insert/update/delete with column values). It is enabled by setting `wal_level = logical` in `postgresql.conf` (requires a restart) and having enough `max_replication_slots` and `max_wal_senders`. **Output plugins.** Logical decoding needs an **output plugin** that formats the decoded changes: - **`pgoutput`** — built into PostgreSQL since v10, no extension install needed; the default/recommended choice for Debezium. It is the same plugin native logical replication uses. - **`wal2json`** — a separate extension producing JSON; was popular before pgoutput matured. - **`decoderbufs`** — Protobuf-based, maintained by the Debezium team; needs installing. The choice is set via `plugin.name`. **Publications (pgoutput).** `pgoutput` filters tables via a **publication** — a Postgres object (`CREATE PUBLICATION`) listing the tables whose changes are streamed. Debezium can auto-create it (`publication.autocreate.mode` = `all_tables` / `disabled` / `filtered`) or you create it manually for least privilege. `publication.name` and `slot.name` name these objects. **Replication slots.** A **replication slot** is a durable, server-side marker that tracks the **LSN (Log Sequence Number)** — a monotonically increasing pointer into the WAL — up to which a consumer has confirmed receipt. Its purpose: **Postgres will not recycle/delete WAL segments that the slot still needs**, guaranteeing the connector never loses changes even across restarts. The slot's `confirmed_flush_lsn` advances as Debezium acknowledges processed LSNs. This is what makes resumption after a connector restart safe and exactly-positioned. **The operational hazard — disk fill.** The same retention guarantee is a double-edged sword: **if the connector is stopped, crashed, or badly lagging, the slot's confirmed LSN stops advancing, and Postgres must keep every WAL segment since that LSN.** `pg_wal` then grows without bound and can **fill the disk and crash the database**. This is the single most common production incident with Postgres CDC. Mitigations: - Monitor `SELECT slot_name, confirmed_flush_lsn, pg_size_bytes(pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)) AS retained FROM pg_replication_slots;` - Alert on retained WAL / slot lag. - **Drop slots you no longer use** (`pg_drop_replication_slot`) — an abandoned slot silently fills the disk. - Use `max_slot_wal_keep_size` (PG 13+) to cap retained WAL — but note this *invalidates* the slot if exceeded, meaning the connector then needs a re-snapshot. Choose the failure mode deliberately. - Debezium's **heartbeat** (`heartbeat.interval.ms`) emits periodic events so the slot advances even on low-traffic or filtered databases; otherwise a slot on an idle/filtered DB can lag because Debezium only acks LSNs it has seen relevant changes for. **Failover caveat.** Before PG 16, logical replication slots did **not** survive a physical failover to a standby (slots weren't replicated). After a failover the slot is gone and a re-snapshot is required. PG 16+ adds failover slots, and Patroni/managed services have their own handling — worth confirming for HA setups. **REPLICA IDENTITY.** To get full `before` images for updates/deletes, the table needs `ALTER TABLE ... REPLICA IDENTITY FULL`; otherwise only primary-key columns appear in `before`.