Why does re.split return the separators when the split pattern contains a capture group?

### The behaviour `re.split(pattern, string, maxsplit=0, flags=0)` cuts `string` at every non-overlapping match of `pattern` and returns the pieces. The wrinkle that surprises people is the interaction with capturing groups: **if the pattern contains capturing groups, the captured text is included in the result list**, interleaved between the fields. ```pycon >>> import re >>> re.split(r"\s*([,;])\s*", "gene1 , gene2; gene3") ['gene1', ',', 'gene2', ';', 'gene3'] >>> re.split(r"\s*(?:[,;])\s*", "gene1 , gene2; gene3") ['gene1', 'gene2', 'gene3'] ``` Both patterns split at the same places. The only difference is whether the delimiter's parentheses capture. Note also that only the *captured* part is inserted — the surrounding whitespace, matched outside the group, is consumed and discarded either way. The list therefore alternates field, capture(s), field, and with one capturing group its length is `2n + 1` for `n` splits. ### Why the design is right The interleaved form makes the split **lossless**. Fields and separators together are exactly the original string, so `"".join(parts)` reconstructs it. That is how you edit fields while preserving whatever separators the input actually used — a tab here, a comma-space there — instead of normalising them on the way out. Tokenisers built on `re.split` rely on it: you get the tokens and the punctuation that joined them in one pass. If the pattern has more than one capturing group, all of them appear, in numeric order, between each pair of fields. A group inside an alternation branch that was not taken did not capture anything, so it contributes `None`: ```pycon >>> re.split(r"(a)|b", "1a2b3") ['1', 'a', '2', None, '3'] ``` Code that assumes "every odd index is a separator string" then hits `None` and fails on the second delimiter kind. Either give every branch a capture, or filter, or restructure the pattern. ### `(?:...)` — grouping without capturing Parentheses in a regex do two jobs at once: they group sub-patterns so a quantifier or an alternation applies to the whole thing, and they capture. `(?:...)` performs only the first. Use it whenever you need the grouping but not the text — which, in a split pattern, is almost always. Non-capturing groups matter beyond `re.split` for a reason that shows up in maintenance. Group numbers follow opening parentheses, so inserting a capturing group in the middle of a pattern renumbers every group after it: `m.group(3)` now points somewhere else, and a `\2` backreference means a different thing. Every group you *do not* capture is one fewer thing that can shift. The corollary is the practical rule: capture what you intend to read, group everything else with `(?:...)`, and name the captures you keep so an edit cannot silently repoint them. ### The other knobs `maxsplit` bounds the number of splits and returns the remainder as the final element: `re.split(r"[,;]", "a,b;c", maxsplit=1)` gives `['a', 'b;c']`. A leading or trailing delimiter produces an empty string at that end of the list — `re.split(r",", ",a,")` is `['', 'a', '']` — which differs from nothing at all and is a common off-by-one in field parsing. Since Python 3.7, a pattern that can match the empty string is allowed in `re.split` (it raised `ValueError` before), which makes splitting on zero-width assertions such as a word boundary possible. ### When not to reach for it `re.split` earns its keep on multi-character, alternating or irregular delimiters. If the delimiter is a fixed substring, `str.split` is faster and clearer; if the format is CSV with quoting rules, a regex is the wrong tool entirely and will fail on a quoted comma. Pick `re.split` when the delimiter genuinely varies, and keep the pattern non-capturing unless you actually want the separators back. ### In an interview Answer in one sentence — capturing groups are included in the result on purpose, so the split round-trips — then show the `(?:...)` fix, then mention the `None` for a non-participating group. That last detail is the one that shows you have debugged this, not just read the note in the docs.