In jOOQ's code generator, how do the includes and excludes patterns decide what gets generated, for example to skip a migration history table?
answer
- two regexes, one outranks the other
- excludes are checked first
- whole-name match, not substring
- default flags ignore whitespace and case
basics
~20 sjOOQ's includes and excludes are Java regular expressions matched against each object's whole name, qualified or not; excludes win over includes, and an empty includes means everything. Excluding flyway_schema_history keeps Flyway's history table out of the generated code.
solid answer
~40 sBoth `<includes>` and `<excludes>` on the generator's `<database>` element are Java regexes, with `|` separating alternatives. Each table, sequence, routine and other object is tested: an exclude match drops it at once, otherwise an include match keeps it, and an empty `includes` means `.*`. The test is `Matcher.matches()`, a whole-string match, tried against the object's unqualified, partially qualified and fully qualified names (partial qualification is on by default), so `flyway_schema_history` or `public\.flyway_schema_history` works while `history` matches nothing. The default `regexFlags` are `COMMENTS CASE_INSENSITIVE`: case is ignored, whitespace in the pattern is ignored and `#` starts a comment. Columns are not filtered unless `includeExcludeColumns` is `true`.
code
xml · 8 lines<database>
<name>org.jooq.meta.postgres.PostgresDatabase</name>
<inputSchema>public</inputSchema>
<excludes>
flyway_schema_history
| payment_import_staging
</excludes>
</database>go deeper
Recall that includes and excludes are regexes on the generator's database element and that excludes win when both match.
Explain whole-name matching, the partially qualified names tried, the default COMMENTS CASE_INSENSITIVE flags, and why columns are untouched by default.
Debug a filter that does not bite by reading the generator's include and exclude log lines and checking the pattern against the full object name.
Decide what the generated layer may expose: excluding history, staging and foreign-owned tables keeps application code from coupling to objects it should never query.
## What includes and excludes are for jOOQ's code generator reads every object its **schema source** reports: tables, views, sequences, routines, user-defined types. Not all of them belong in application code. A migration tool's **history table** (Flyway's is `flyway_schema_history` by default), audit or staging tables, and objects owned by another service are the usual candidates to leave out. Two elements on the generator's `<database>` configuration do the filtering: - `<includes>`: what to generate. - `<excludes>`: what to drop, even if it is included. Both hold a **Java regular expression**; alternatives are separated with `|`. ## The algorithm, as the source implements it `AbstractDatabase.filterExcludeInclude` applies the same steps to each object: 1. If no `includes` is configured, it becomes `.*`: include everything. 2. Test every exclude pattern first. A match drops the object immediately. 3. Otherwise test the include patterns. The first match keeps the object. 4. Objects matching no include pattern are dropped. So **excludes have priority**: an object matching both is not generated. The XSD documentation says the same: "Excludes match before includes, i.e. excludes have a higher priority." ## What a pattern is matched against Two details trip people up. - **Whole-string matching.** The check uses `Pattern.matcher(name).matches()`, not `find()`. The pattern must describe the entire name. `history` does not match `flyway_schema_history`; `.*history` does. - **Several spellings of the name.** With `regexMatchesPartialQualification` at its default of `true`, the pattern is tried against each partially qualified name, from the bare name `flyway_schema_history` up to the fully qualified name, such as `public.flyway_schema_history`. Any match counts. With it set to `false`, only the bare and the fully qualified names are tried. Because `.` is a regex metacharacter, write a qualified name as `public\.flyway_schema_history`; an unescaped dot matches any character and may match more than intended. ## The default regex flags `<regexFlags>` defaults to `COMMENTS CASE_INSENSITIVE`, applied to every regex in the configuration: | Flag | Effect on a pattern | |---|---| | `CASE_INSENSITIVE` | `PAYMENT` and `payment` both match `payment` | | `COMMENTS` | whitespace in the pattern is ignored, `#` starts a comment to end of line | The `COMMENTS` flag lets you lay a long pattern over several lines, but it also means a literal space or `#` in a name must be escaped. If you override `regexFlags`, you lose both defaults unless you list them again. ## Allow-list or deny-list Two styles work, and teams should pick one on purpose: - **Deny-list**: leave `includes` at `.*` and grow `excludes` as unwanted objects appear. New tables are generated automatically, which suits a service that owns its whole schema. - **Allow-list**: name the wanted objects in `includes`, for example `payment | refund | ledger_entry`. Nothing new reaches application code unless someone adds it, which suits a schema shared with other services, at the price of editing the pattern for each new table. ## Scope: objects, not columns By default the patterns filter top-level objects only. Two switches widen them: - `includeExcludeColumns` (default `false`) also filters columns inside tables. - `includeExcludePackageRoutines` (default `false`) also filters routines inside packages. Restricting whole schemas is a separate job: `<inputSchema>` names one schema to read, and left empty, all available schemas are generated. ## Migration history tables per schema source | Schema source | History table in output? | |---|---| | Live database migrated by Flyway | yes, until you exclude it | | `LiquibaseDatabase` | no: it removes Liquibase's changelog and lock tables unless `includeLiquibaseTables` is `true` | | `DDLDatabase` over migration scripts | no table to exclude: the scripts do not create one | ## A working configuration ```xml <database> <name>org.jooq.meta.postgres.PostgresDatabase</name> <inputSchema>public</inputSchema> <includes>.*</includes> <excludes> flyway_schema_history # migration bookkeeping | payment_import_staging # loaded by a batch job, not the service </excludes> </database> ``` The whitespace and the `#` comments are legal only because of the default `COMMENTS` flag. ## How to check the result - Run the generator with debug logging: `filterExcludeInclude` logs "Excluding ... because of pattern ..." and "Including ... because of pattern ..." for each object. - Look for the table class in the generated `tables` package; a history table that still appears means the pattern did not match the whole name.
- Why did an excludes value of history leave Flyway's flyway_schema_history table in jOOQ's generated code?jOOQ tests patterns with `Matcher.matches()`, which must match the whole name, so `history` matches only an object named exactly `history`. Write the full name, `flyway_schema_history`, or a pattern that covers the prefix, such as `.*history`. Debug logging shows which pattern included or excluded each object.
- Does a jOOQ excludes pattern remove a column such as an internal audit column?Not by default. includes and excludes filter top-level objects such as tables, sequences and routines; columns are filtered only when `includeExcludeColumns` is set to `true`, and then the same patterns are tried against column names too, so a broad pattern can start dropping columns you meant to keep.
A door list with a banned list checked first: the bouncer turns away anyone on the banned list even if they are also on the guest list, and a name only counts when it matches the whole name on the badge, not part of it.
- An excludes pattern of history removes every table whose name contains history.
- When an object matches both patterns, includes wins and it is generated.
- jOOQ's codegen regexes are case-sensitive unless you add (?i).
- excludes also removes matching columns, with no extra setting.
- The generator matches patterns only against fully qualified names.