When should the repl argument to re.sub be a callable instead of a replacement string?

### Two shapes of the same argument `re.sub(pattern, repl, string, count=0, flags=0)` accepts either a string or a callable as `repl`, and the two are genuinely different mechanisms rather than a convenience pair. **As a string**, `repl` is a template with its own small grammar. Backslash-digit (`\1` … `\99`) inserts the text of that capturing group; `\g<name>` and `\g<1>` are the explicit forms; `\g<0>` is the whole match. Standard escapes such as `\n` are honoured, `\\` is a literal backslash, and an unknown escape of an ASCII letter is an error. Everything else is copied literally. That is all a template can do: rearrange and interleave captured text with constants. ```pycon >>> import re >>> re.sub(r"(\w+)@(\w+)", r"\2:\g<1>", "ada@lab") 'lab:ada' ``` **As a callable**, `repl` is invoked once per match, in order, with a single argument: the Match object. Its return value must be a `str` — return an int and you get `TypeError: sequence item 1: expected str instance, int found`, which is a confusing message the first time you see it, because the failure surfaces when re joins the pieces. The returned string is inserted **verbatim**: nothing in it is scanned for `\1` or any other escape. ### When you need the callable Any replacement that is a function of the match: * **Arithmetic on captured text** — shift every coordinate in a report by one, renumber a suffix. * **A lookup** — map a matched identifier through a dict, falling back to leaving it alone. * **A conditional** — some matches change, others do not. `return m.group(0)` is the idiomatic no-op; it is cheaper and clearer than trying to make the pattern exclude the cases you want to skip. * **Text that came from data** — a literal replacement built from a value you did not write is unsafe as a template, because a backslash in it is interpreted; a callable returning that value sidesteps the whole grammar. ```python import re def bump(m): return f"{m.group('gene')}_{int(m.group('n')) + 1}" re.sub(r"(?P<gene>[A-Z0-9]+)_(?P<n>\d+)", bump, "BRCA_1 TP53_7") # 'BRCA_2 TP53_8' ``` ### Call semantics worth knowing The function runs once per match, not once per input string, and it runs during the scan — so it sees matches in left-to-right order and can carry state between them. `count` limits how many matches are replaced (0 means all), so the callable is not necessarily called for every match in the text. The original string is never mutated: `str` is immutable and `re.sub` returns a new one. `re.subn` does the same work but returns `(new_string, number_of_substitutions)`, which is the honest way to assert "exactly one edit happened" instead of comparing strings afterwards. ### The stateful-callable trap Because the callable can carry state, people reach for the shortest way to give it some — a default argument: ```python def label(m, seen={}): # do not do this seen[m.group(0)] = seen.get(m.group(0), 0) + 1 return f"{m.group(0)}#{seen[m.group(0)]}" ``` Default values are evaluated **once, when the `def` executes**, and that one dict is shared by every call for the life of the process. In a nightly genome-annotation run that walks thousands of files over six hours, the counter never resets between files: file two starts at whatever number file one ended on, and the corruption is invisible until someone diffs the output. The fixes are all ordinary Python — build a fresh closure per file (`def make_labeller(): counts = {}; def label(m): ...`), use a small class with `__call__`, or pass the state explicitly with `functools.partial(label, counts=fresh_dict)`. The rule of thumb: if a `repl` callable has state, make the state's lifetime a visible, per-call decision. ### Choosing between the forms Prefer the string template when the replacement really is "these captures in a different arrangement" — it is shorter, it is obvious to a reader, and it avoids a Python call per match. Prefer the callable as soon as a conditional or a computation appears, or as soon as any part of the replacement text comes from data. And when neither a pattern nor a computation is involved at all — a fixed substring swapped for another — `str.replace` is faster and says exactly what it does. ### In an interview The answer that lands is: string repl is a template that can only reference groups; a callable receives the Match and returns the replacement, so it handles anything computed or conditional; `m.group(0)` means "leave this one". Adding the `re.subn` count and the "no escape processing on the return value" detail is what separates a used-it answer from a read-about-it answer.