How do you restrict which tables and columns a Debezium connector emits change events for?

## The options Debezium scopes capture with paired include/exclude properties at several levels: - `table.include.list` / `table.exclude.list` — which tables produce change events. - `schema.include.list` / `schema.exclude.list` on Postgres, SQL Server and Oracle; `database.include.list` / `database.exclude.list` on MySQL — a coarser filter above tables. - `column.include.list` / `column.exclude.list` — which fields appear inside each event. - `column.mask.with.<length>.chars` and hash-based masking — keep the column in the event, replace its value. For any pair you set **one** side. Configuring both the include and the exclude form of the same option is a configuration error, not a merge. ## Matching rules — the part candidates get wrong Values are comma-separated **regular expressions**, and Debezium matches each against the *entire* fully-qualified identifier, not as a substring search. The identifier's shape depends on the engine: MySQL has no schema layer, so it is `databaseName.tableName`; Postgres, SQL Server and Oracle use `schemaName.tableName`. The consequences are easy to demonstrate. `table.include.list=public.order` captures `public.order` and nothing else — not `public.order_items`, because the match is against the whole name. To capture the family you write `public.order.*`. Conversely a pattern like `.*orders.*` matches far more than intended once a second schema appears. Because `.` is a regex metacharacter as well as the separator, `public.orders` also technically matches `publicXorders`; harmless in practice, but a reason to be explicit rather than clever with these patterns. Column lists use three-part names — `schema.table.column` (or `database.table.column` on MySQL) — so a bare column name never matches. ## Filtering happens in the connector, not in the database This is the operationally important point. The include list tells Debezium which events to *emit*; it does not by itself tell the source database to send less. On PostgreSQL, what the database decodes and streams is determined by the **publication**. Debezium's `publication.autocreate.mode` controls that: - `all_tables` — create a publication covering every table (requires elevated privileges), so the connector receives all changes and throws away the ones outside its include list. - `filtered` — create a publication restricted to the tables matching the include configuration, so the database itself sends only what you want. - `disabled` — assume a publication already exists, managed by a DBA. On a busy cluster where you capture three tables out of four hundred, `filtered` (or a hand-managed publication) is the difference between decoding and shipping everything and decoding almost nothing. It also matters for privileges: creating an all-tables publication needs rights many production databases will not grant. MySQL has an analogous cost: the binlog is server-wide, so the connector reads every event and discards non-matching ones. There is no server-side filter, but you can at least stop Debezium from tracking DDL for tables you do not capture, which keeps schema history small. ## Column filtering and its traps Excluding columns is attractive for PII and for very wide tables, but two traps recur. First, **never exclude a primary-key column**: Debezium builds the event key from the key columns, and downstream sinks use that key for upserts, compaction and ordering. Remove it and you have events nobody can apply. Second, exclusion is not redaction at the source — the value still travelled from the database into the connector's memory; it is simply not written to the event. When the requirement is 'the value must never leave the database in any readable form', masking or hashing the column is the honest answer, and column-level access control at the source is stronger still. Masking with `column.mask.with.<length>.chars` replaces a value with a fixed-length string of asterisks; hash-based masking with a salt is available when you need the masked value to remain join-able across events without being reversible. ## Changing the lists later Adding a table to the include list does **not** backfill it. The connector begins streaming that table's *future* changes; every row that existed beforehand is missing until you take a snapshot of the newly-included table — which is what Debezium's snapshot and signalling features exist for. Plan the addition as a two-step operation: change the configuration, then backfill. Removing a table is simpler, but consumers of its topic will simply see the stream go quiet, so tell them. ## Interview framing A good answer names the properties, states the whole-identifier matching rule with an example, and then volunteers the two production consequences: connector-side filtering versus database-side scoping, and the need to backfill anything newly included.