Does the same exclusion regex behave identically in every OWASP ZAP layer that reads it?
answer
- one property is shared, the rest are not
- two call sites reached for a different API
- one list, more than one reader
- the shipped defaults all start the same way
- an inline flag compensating for a missing one
basics
~20 sNo. Only the full-match anchoring is shared. A context matches case-insensitively with the query already removed, the proxy handler and the crawler match the whole URI case-sensitively, and the active scanner matches that same shared list case-insensitively.
solid answer
~40 sThe layers agree on one thing only: every match is a **full** match, never a search. Beyond that a context compiles its entries with the case-insensitive flag and matches them against the URL with everything after the first `?` removed; the `network` add-on's proxy handler and the `spider` add-on's fetch filter both call `String.matches`, which is **case-sensitive**, against the whole URI including the query; and core's active scanner compiles its exclude entries with the case-insensitive flag. The bite is that the global exclusion list is read by all three of those consumers, so one entry is case-sensitive for two of them and case-insensitive for the third. The project works around it in its own defaults: every shipped global exclusion pattern starts with an inline `(?i)`.
go deeper
Recall that these patterns are matched against the whole string, not searched for inside it, so a bare path will not work anywhere.
Say which layer removes the query string and which does not, and be able to explain why the same pattern can fire in one place and not another.
Know that case sensitivity differs by call site on a shared list, and recognise the inline flag in the shipped defaults as the project's own workaround for it.
Set the convention that closes this whole class: patterns are written for one named layer, carry their own inline flag, and are verified by what the run did rather than by reading them back.
## The same string, four different questions OWASP ZAP asks "does this regex match this URL?" in several places, and the places do not agree on what the regex is matched against, whether the match is anchored to the whole string, or whether case matters. Measured against the source, the scope-related layers look like this: | layer | applied to | anchoring | case | |---|---|---|---| | a context's include and exclude lists (core `Context`) | the URL, truncated at the first `?` | full match | **insensitive** | | the local proxy handler (`network` add-on) | the whole request URI, query included | full match | **sensitive** | | the crawler's fetch filter (`spider` add-on) | the whole URI, query included | full match | **sensitive** | | the active-scan exclude list (core `Scanner`) | the whole URI, query included | full match | **insensitive** | The anchoring is the one thing they all share: every one of these is a full match, so a pattern must describe the entire string it is handed. Nothing here is a search. ## Where the difference comes from It is not a design decision stated anywhere; it falls out of which API each call site reached for. - The context layer compiles each entry once with the case-insensitive flag and keeps the compiled `Pattern`. - The proxy handler and the crawler's filter call `String.matches(pattern)`, which compiles with no flags at all — and is therefore **case-sensitive**. - Core's active scanner compiles its exclude entries itself, and sets the case-insensitive flag when it does. ## The consequence that matters The **same list** can be read by more than one of those call sites. The global exclusions are handed to the proxy handler, to the crawler's filter, and to the active scanner. So one entry in one list is case-sensitive for two of its readers and case-insensitive for the third. A pattern written as `https://example\.com/Admin.*` keeps `/Admin` away from the active scanner *and* from `/admin` — but at the proxy and at the crawler it will not match `/admin` at all. The project knows this, and you can see the workaround in its own defaults: **every shipped global exclusion pattern begins with the inline flag `(?i)`.** That prefix is not stylistic. It is what makes those patterns behave the same way at all three readers, and it is the cheapest evidence available that the divergence is real. ## The other divergence: what string is handed over The context layer is the only one that removes the query string before matching. Everywhere else the full URI is matched, question mark and parameters included. That has two symmetrical consequences, and both of them are worth saying out loud: 1. **A pattern keyed on a query parameter can never match a context rule.** The text is gone. 2. **A pattern keyed on a query parameter is exactly right for a global exclusion**, and the shipped defaults prove it: several of them contain an escaped `\?` and match on what follows it. Copy one of those patterns into a context's `excludePaths` and it becomes inert — legal, accepted, and incapable of firing. ## It happens inside a single method, twice This is not only a difference between distant components. The crawler's fetch filter, on one URI in one call, asks its bound context (query removed, case-insensitive) and then applies its exclude list (whole URI, case-sensitive). Core's active-scan scope check does the same shape of thing: it consults the session's derived scope, which strips the query, and then applies its own exclude patterns to the full URI. **One method, two rules, on the same string.** ## What to do about it - **Do not move a regex between layers.** Rewrite it for the layer it is going into, and expect the anchoring to be the only property it keeps. - **Put `(?i)` at the front of any exclusion pattern you write for a list-based layer**, exactly as the shipped defaults do. It costs nothing where the layer was already case-insensitive. - **Keep query-keyed patterns out of contexts** and path-keyed patterns as the only thing a context carries. That is not a preference; it is the only shape a context can act on. - **Test a new exclusion by observing what the run did**, not by reading the pattern back. Every one of these failures is silent: no validation rejects a pattern for being written against the wrong string, because as a regular expression it is perfectly well formed.
- Why does every shipped global exclusion pattern begin with `(?i)`?Because two of the three components that read that list compile the pattern with no flags, which makes it case-sensitive, while the third sets the case-insensitive flag itself. The inline prefix is the only way to make one entry behave the same at all of its readers, and it is harmless where the flag was already set.
- Is a scope or exclusion regex ever searched for inside the URL rather than matched in full?Not at any of these layers — every one of them anchors to the whole string, which is why a bare path never works and why a trailing `.*` is the habit worth having. Other parts of the program do use a search, but they are matching things other than a URL and deciding something other than scope, so treat full match as the rule here and check any exception against the source before relying on it.
- What single property is common to all of these layers, and why does it matter?They all call a full match, so the pattern must describe the entire string it is handed. That is why a bare path never works anywhere and why a trailing `.*` is the habit worth having — it is the one thing you can carry between layers without rewriting.
saying these in an interview costs you the question
- Assumes a regex that works in one exclusion setting works in all of them
- Says every regex in the program is matched case-insensitively
- Expects a context exclude pattern to be shown the query string
- Copies a shipped global-exclusion pattern into a context and expects it to fire
- Thinks these exclusion patterns are searched for inside the URL rather than matched in full