Capture Groups and Substitution

### What creates a group Every pair of parentheses in a Python regular expression that is not one of the `(?...)` extension forms creates a **capturing group** — a slot in which the engine records the substring that part of the pattern consumed. Groups are numbered by the position of their *opening* parenthesis, scanning left to right, starting at 1. Group 0 is reserved for the entire match. Nesting does not change the rule: in `((a)(b))` the outer group is 1, `(a)` is 2 and `(b)` is 3. Counting parentheses by hand is exactly what goes wrong on any pattern longer than one line, which is the practical argument for naming. ### Naming: `(?P<name>...)` Python spells a named group `(?P<name>...)`. The `P` is Python's own marker on the shared `(?...)` extension syntax — the .NET/PCRE spelling `(?<name>...)` is a syntax error here, and getting that wrong is one of the most common small mistakes in an interview. The name must be a valid Python identifier and must be unique within the pattern; reusing one raises `re.error` at compile time ("redefinition of group name"). A named group is *still* a numbered group: naming is additive, so `\1` and `m.group(1)` keep working alongside `m.group("name")`. ### The accessors Given a match object `m`: * `m.group()` and `m.group(0)` return the whole matched substring. * `m.group(n)` returns group `n`; `m.group("name")` returns a named group. Passing several indexes, `m.group(1, 3)`, returns a tuple. * `m.groups()` returns a tuple of **all capturing groups, 1 through N** — group 0 is not in it, and neither are non-capturing `(?:...)` groups, which record nothing. * `m.groupdict()` returns `{name: text}` for the **named groups only**. A pattern with no named groups gives an empty dict. * `m.lastindex` and `m.lastgroup` name the last group that matched, which is occasionally useful for alternations. * On the compiled pattern, `.groupindex` is a mapping of name to group number. ```python import re m = re.search(r"(?P<chrom>chr\d+):(?P<start>\d+)-(?P<end>\d+)", "chr7:117120017-117308719") m.group("chrom") # 'chr7' m.group(2) # '117120017' — same group as 'start' m.groups() # ('chr7', '117120017', '117308719') m.groupdict() # {'chrom': 'chr7', 'start': ..., 'end': ...} ``` ### Groups that did not participate An optional group, or a group inside an alternation branch the engine did not take, captured nothing. Python reports that as `None`, not `""`. This is the single most common source of `TypeError: int() argument must be...` in parsing code: `int(m.group("version"))` blows up on the record that omitted the suffix. Both `groups()` and `groupdict()` accept a default to substitute instead — `m.groups("")`, `m.groupdict("")` — and a bare `m.group("version")` still returns `None`, so a guard or a default is your choice per call site. ```python m = re.fullmatch(r"(\w+?)(?:\.(\d+))?", "BRCA1") m.groups() # ('BRCA1', None) m.groups("-") # ('BRCA1', '-') ``` Note the `(?:...)` around the optional suffix in that pattern: it groups `\.(\d+)` so the `?` applies to the whole thing, without adding a capture slot of its own. ### Referring to a group from inside the pattern Groups can be referenced later in the same pattern — a **backreference** — numerically as `\1` or by name as `(?P=name)`. That is how you assert that two parts of the text are identical, such as a repeated word or a closing tag matching its opener: ```python re.findall(r"<(?P<t>\w+)>.*?</(?P=t)>", "<a>1</a><b>2</b>") # ['a', 'b'] ``` ### Why names are worth the extra characters Three practical reasons. First, edits: add one group near the front of a pattern and every numeric index after it shifts, silently breaking `m.group(3)` and any `\2` downstream — names are immune. Second, readability at the call site: `m.group("start")` says what it means where `m.group(2)` needs the reader to go count. Third, `m.groupdict()` drops straight into a record — in a text-extraction pipeline over annotation lines, `groupdict()` is often the row you were going to build by hand anyway. The cost is that a named group is a capturing group, so where you only need grouping for a quantifier or an alternation, use `(?:...)` and keep the numbering clean. ### In an interview Say the three things out loud: numbering starts at 1 and 0 is the whole match; `groups()` is a tuple of all captures while `groupdict()` is a dict of named ones; non-participating groups are `None`. That is the whole answer, and offering the `None` detail unprompted is what makes it sound like you have parsed real text.