A PHP worker extracting order references from large support emails with preg_match() silently finds none on some messages; how do pcre.backtrack_limit and preg_last_error() explain it?

seniorshould knowfreq 30%

answer

a work budget per match
pcre.backtrack_limit defaults to 1000000
false or null, no warning
PREG_BACKTRACK_LIMIT_ERROR
JIT stack limit is a separate error

basics

~10 s

When a match needs more work than pcre.backtrack_limit (1000000 by default) allows, preg_match() returns false without a warning and preg_last_error() reports PREG_BACKTRACK_LIMIT_ERROR; code that tests the result for truthiness reads that as 'no reference'.

solid answer

~40 s

PCRE caps how much work one match attempt may do. PHP passes pcre.backtrack_limit, default 1000000, as PCRE2's match limit and pcre.recursion_limit, default 100000, as its depth limit; with pcre.jit=1, the default, a JIT-compiled pattern can also exhaust its fixed JIT stack. When a limit is hit on a long email with many near-matches, preg_match() and preg_match_all() return false, preg_replace() returns null and preg_split() returns false, without any warning. preg_last_error() then returns PREG_BACKTRACK_LIMIT_ERROR, PREG_RECURSION_LIMIT_ERROR or PREG_JIT_STACKLIMIT_ERROR, and preg_last_error_msg() gives text such as Backtrack limit exhausted. An if (preg_match(...)) check turns this into a silent miss. Fix it in three layers: check === false and log the error, rewrite the pattern so it cannot backtrack heavily, and bound the input, for example by stripping quoted replies. Raising the limit only buys time.

PCRE caps how much work one match attempt may do. PHP passes `pcre.backtrack_limit`, default `1000000`, as PCRE2's match limit and `pcre.recursion_limit`, default `100000`, as its depth limit; with `pcre.jit=1`, the default, a JIT-compiled pattern can also exhaust its fixed JIT stack. When a limit is hit on a long email with many near-matches, `preg_match()` and `preg_match_all()` return `false`, `preg_replace()` returns `null` and `preg_split()` returns `false`, **without** any warning. `preg_last_error()` then returns `PREG_BACKTRACK_LIMIT_ERROR`, `PREG_RECURSION_LIMIT_ERROR` or `PREG_JIT_STACKLIMIT_ERROR`, and `preg_last_error_msg()` gives text such as `Backtrack limit exhausted`. An `if (preg_match(...))` check turns this into a silent miss. Fix it in three layers: check `=== false` and log the error, rewrite the pattern so it cannot backtrack heavily, and bound the input, for example by stripping quoted replies. Raising the limit only buys time.

code

php · 22 lines

php

<?php
declare(strict_types=1);

function findOrderRef(string $messageId, string $body): ?string
{
    $result = preg_match('/\bORD-[0-9]{4}-[0-9]{6}\b/', $body, $m);

    if ($result === false) {
        error_log(sprintf(
            'order-ref regex failed for %s: %d %s',
            $messageId,
            preg_last_error(),
            preg_last_error_msg(),
        ));
        throw new RuntimeException('Order reference scan failed');
    }

    return $result === 1 ? $m[0] : null;
}

var_dump(ini_get('pcre.backtrack_limit'));  // string(7) "1000000" unless configured
var_dump(ini_get('pcre.jit'));              // string(1) "1" unless configured

go deeper

for a junior

Recall that preg_match can return false and that preg_last_error_msg explains why.

for a middle

Explain pcre.backtrack_limit, pcre.recursion_limit and pcre.jit, and which failure value each preg_* function returns when a limit is hit.

for a senior

Diagnose silent extraction misses from logs, reproduce with the same ini settings, then fix the pattern and bound the input rather than raising the limit.

for a principal

Set a policy for regex on untrusted input: strict failure checks, bounded input sizes, pattern review, and CPU budgets per message in workers.

The symptom

A queue worker reads inbound support emails and attaches each to an order by finding a reference like ORD-2026-000123. Most emails work. A few, typically very long threads with quoted replies, HTML remnants or base64 blocks, are filed as "no order reference" even though a human can see one. Nothing appears in the error log.

Why PCRE gives up

PHP's PCRE extension runs patterns on the PCRE2 library, with limits that stop a single match from running away:

Directive	Default	What PHP does with it
`pcre.backtrack_limit`	`1000000`	passed as PCRE2's match limit, a cap on internal matching work
`pcre.recursion_limit`	`100000`	passed as PCRE2's depth limit
`pcre.jit`	`1`	compiles patterns to machine code, which run on a JIT stack of fixed maximum size

The defaults come from the extension source. The commented-out pcre.backtrack_limit=100000 line in the shipped php.ini-production is a historical value and does not reflect the default; ini_get('pcre.backtrack_limit') returns 1000000 unless you set it.

A pattern with ambiguous repetition, such as nested quantifiers or .* followed by more pattern, can try an enormous number of ways to match a long near-miss. Why that happens is regex-engine theory; the PHP-specific part is what the functions do when the budget runs out.

What the functions return

preg_match(), preg_match_all(), preg_split() and preg_grep() return false.
preg_replace(), preg_replace_callback() and preg_filter() return null.
No warning or exception is raised for these runtime limits.
preg_last_error() returns PREG_BACKTRACK_LIMIT_ERROR, PREG_RECURSION_LIMIT_ERROR or PREG_JIT_STACKLIMIT_ERROR.
preg_last_error_msg() returns Backtrack limit exhausted, Recursion limit exhausted or JIT stack limit exhausted.

With if (preg_match(...)), false is falsy, and the email is treated as having no reference. With $body = preg_replace(...), the body becomes null, and the damage appears later and somewhere else.

Diagnosing it

Make every preg_* call on external input check its failure value strictly and log preg_last_error() with preg_last_error_msg() and the email's identifier.
Reproduce with the logged email and the same PHP configuration, including pcre.jit, because JIT and interpreter hit different limits.
Check ini_get('pcre.backtrack_limit') in the worker, since CLI and web configurations often differ.
Look at the pattern's shape around repetition and at how large the subject is.

Fixing it

Rewrite the pattern so each position can match in only one way: anchor it, replace .* with a precise class such as [0-9]{6}, and use possessive quantifiers or atomic groups where backtracking cannot help. A reference pattern like /\bORD-[0-9]{4}-[0-9]{6}\b/ has almost nothing to backtrack over.
Bound the input: strip quoted replies and attachments first, or scan only the part of the message a reference can appear in.
Treat the limit as a guard, not a knob: ini_set('pcre.backtrack_limit', ...) works at runtime, but raising it lets a bad pattern consume more CPU per message, which in a worker processing untrusted mail is a denial-of-service risk.

Other silent failures with the same shape

The same false/null contract covers PREG_BAD_UTF8_ERROR, when a subject with invalid UTF-8 meets the u modifier, and PREG_BAD_UTF8_OFFSET_ERROR, when $offset points inside a multi-byte character. Email bodies decoded with the wrong charset produce the former, so a single strict check and log line catches both classes of problem.

What interviewers listen for

This is a production-diagnosis question. Interviewers want to hear, in order:

that the silence itself is the clue, because runtime PCRE limits do not warn;
the three directives and their real defaults, including the misleading commented line in php.ini-production;
the failure value per function, false for matching and splitting, null for replacing;
a fix that changes the pattern and the input size rather than only the limit;
a monitoring change, so the next occurrence produces a log line with the error code instead of a quietly misfiled ticket.

Candidates who jump straight to ini_set() usually get a follow-up about denial of service.

## The symptom A queue worker reads inbound support emails and attaches each to an order by finding a reference like `ORD-2026-000123`. Most emails work. A few, typically very long threads with quoted replies, HTML remnants or base64 blocks, are filed as "no order reference" even though a human can see one. Nothing appears in the error log. ## Why PCRE gives up PHP's PCRE extension runs patterns on the PCRE2 library, with **limits** that stop a single match from running away: | Directive | Default | What PHP does with it | |---|---|---| | `pcre.backtrack_limit` | `1000000` | passed as PCRE2's match limit, a cap on internal matching work | | `pcre.recursion_limit` | `100000` | passed as PCRE2's depth limit | | `pcre.jit` | `1` | compiles patterns to machine code, which run on a JIT stack of fixed maximum size | The defaults come from the extension source. The commented-out `pcre.backtrack_limit=100000` line in the shipped `php.ini-production` is a historical value and does **not** reflect the default; `ini_get('pcre.backtrack_limit')` returns `1000000` unless you set it. A pattern with ambiguous repetition, such as nested quantifiers or `.*` followed by more pattern, can try an enormous number of ways to match a long near-miss. Why that happens is regex-engine theory; the PHP-specific part is what the functions do when the budget runs out. ## What the functions return - `preg_match()`, `preg_match_all()`, `preg_split()` and `preg_grep()` return `false`. - `preg_replace()`, `preg_replace_callback()` and `preg_filter()` return `null`. - **No warning or exception** is raised for these runtime limits. - `preg_last_error()` returns `PREG_BACKTRACK_LIMIT_ERROR`, `PREG_RECURSION_LIMIT_ERROR` or `PREG_JIT_STACKLIMIT_ERROR`. - `preg_last_error_msg()` returns `Backtrack limit exhausted`, `Recursion limit exhausted` or `JIT stack limit exhausted`. With `if (preg_match(...))`, `false` is falsy, and the email is treated as having no reference. With `$body = preg_replace(...)`, the body becomes `null`, and the damage appears later and somewhere else. ## Diagnosing it 1. Make every `preg_*` call on external input check its failure value strictly and log `preg_last_error()` with `preg_last_error_msg()` and the email's identifier. 2. Reproduce with the logged email and the same PHP configuration, including `pcre.jit`, because JIT and interpreter hit different limits. 3. Check `ini_get('pcre.backtrack_limit')` in the worker, since CLI and web configurations often differ. 4. Look at the pattern's shape around repetition and at how large the subject is. ## Fixing it - **Rewrite the pattern** so each position can match in only one way: anchor it, replace `.*` with a precise class such as `[0-9]{6}`, and use possessive quantifiers or atomic groups where backtracking cannot help. A reference pattern like `/\bORD-[0-9]{4}-[0-9]{6}\b/` has almost nothing to backtrack over. - **Bound the input**: strip quoted replies and attachments first, or scan only the part of the message a reference can appear in. - **Treat the limit as a guard, not a knob**: `ini_set('pcre.backtrack_limit', ...)` works at runtime, but raising it lets a bad pattern consume more CPU per message, which in a worker processing untrusted mail is a denial-of-service risk. ## Other silent failures with the same shape The same `false`/`null` contract covers `PREG_BAD_UTF8_ERROR`, when a subject with invalid UTF-8 meets the `u` modifier, and `PREG_BAD_UTF8_OFFSET_ERROR`, when `$offset` points inside a multi-byte character. Email bodies decoded with the wrong charset produce the former, so a single strict check and log line catches both classes of problem. ## What interviewers listen for This is a production-diagnosis question. Interviewers want to hear, in order: - that the silence itself is the clue, because runtime PCRE limits do not warn; - the three directives and their real defaults, including the misleading commented line in `php.ini-production`; - the failure value per function, `false` for matching and splitting, `null` for replacing; - a fix that changes the pattern and the input size rather than only the limit; - a monitoring change, so the next occurrence produces a log line with the error code instead of a quietly misfiled ticket. Candidates who jump straight to `ini_set()` usually get a follow-up about denial of service.

Is raising pcre.backtrack_limit a good fix?
Rarely. It is `PHP_INI_ALL`, so `ini_set()` can raise it, but that only lets a pathological pattern burn more CPU per input, which is a denial-of-service risk on untrusted mail. Rewrite the pattern to avoid ambiguous repetition, and bound the input size; keep the limit as the safety net.
Why can the same pattern fail with PREG_JIT_STACKLIMIT_ERROR on one server and PREG_BACKTRACK_LIMIT_ERROR on another?
With `pcre.jit=1` compiled patterns run on a JIT stack of fixed maximum size and can run out of it; with JIT disabled, or unavailable in the build, the interpreter hits the match or depth limit instead. Compare `pcre.jit` and `PCRE_JIT_SUPPORT` on both machines.
What does preg_replace() return when the backtrack limit is exhausted, and why is that dangerous?
It returns `null`. Code such as `$body = preg_replace(...)` replaces the email text with `null`, and the failure surfaces later as an empty body or a `TypeError` far from its cause. Compare the result with `=== null` before assigning.

saying these in an interview costs you the question

Hitting the backtrack limit emits a warning, so it will show up in the logs.
The default pcre.backtrack_limit is 100000, as php.ini-production shows.
When the limit is hit, preg_match returns 0 because nothing was found.
Raising pcre.backtrack_limit is the proper fix for a slow pattern.
The JIT removes all limits, so JIT-enabled servers cannot hit them.

A PHP worker extracting order references from large support emails with preg_match() silently finds none on some messages; how do pcre.backtrack_limit and preg_last_error() explain it?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

The symptom

Why PCRE gives up

What the functions return

Diagnosing it

Fixing it

Other silent failures with the same shape

What interviewers listen for

▾common follow-ups3 questions

▾red flags5 flags

The symptom

Why PCRE gives up

What the functions return

Diagnosing it

Fixing it

Other silent failures with the same shape

What interviewers listen for

answer

go deeper