Why is regexp.MatchString(pattern, s) more expensive than calling MatchString on a compiled *regexp.Regexp?
answer
- one of them returns an error
- what could still fail at match time?
- the wrapper builds and throws away
- fixed cost paid per call, not per byte
- hoist the compiled value, not the string
basics
~20 sThe package-level regexp.MatchString compiles its pattern from scratch on every call and throws the compiled matcher away afterwards. The method runs on a *regexp.Regexp that was compiled once, so each call pays only for matching.
solid answer
~40 s`regexp.MatchString(pattern, s string) (bool, error)` is a convenience wrapper: it calls `regexp.Compile` on the pattern, runs the match, and discards the compiled `*regexp.Regexp`. Nothing is cached between calls, which is also why it returns an error — the pattern can fail to parse on any call. The method `(*regexp.Regexp).MatchString(s string) bool` has no error to return because compilation already happened, once, when you built the value. Parsing and compiling a pattern is much more work than matching one short string, so calling the package-level function inside a loop turns a one-off cost into a per-item cost. The fix is the standard idiom: hoist the pattern into a `var re = regexp.MustCompile(...)` (or compile once at start-up) and call the method in the loop. `regexp.Match` on `[]byte` behaves exactly the same way.
code
go · 11 linesvar reqRe = regexp.MustCompile(`^GET /\S+`)
// Compiled once at package init; this call only matches.
func isGetFast(line string) bool {
return reqRe.MatchString(line)
}
// Parses and compiles the same pattern on every call, then discards it.
func isGetSlow(line string) (bool, error) {
return regexp.MatchString(`^GET /\S+`, line)
}go deeper
Remember the shape: build a *regexp.Regexp once with regexp.MustCompile, then call its MatchString in your code. Reach for regexp.MatchString only for a one-off check.
Explain that the package-level function compiles and discards on every call, that its error exists because parsing happens there, and that compiling is far heavier than matching one short string.
Be able to spot this in review and in a CPU profile, and to say where the compiled value should live when the pattern is only known at runtime rather than written in source.
Own the guideline: which layers may compile patterns at all, and how a codebase keeps per-item compilation out of hot paths as new code is written rather than discovering it in a profile later.
## Two APIs that look alike and cost very differently The `regexp` package exposes both a set of package-level convenience functions and a set of methods on the compiled type: - `func MatchString(pattern string, s string) (matched bool, err error)` — package level. - `func (re *Regexp) MatchString(s string) bool` — method on a compiled value. The first is defined in terms of the second. It compiles `pattern`, and if that succeeds it calls the method on the freshly compiled value and returns the result. The compiled value is then unreachable and collected. `func Match(pattern string, b []byte) (bool, error)` is the `[]byte` twin and works identically. ## Why the wrapper returns an error The error is the giveaway. A method on an already-compiled `*regexp.Regexp` cannot fail to parse anything — parsing is behind it — so `(*Regexp).MatchString` returns a bare `bool`. The package-level function *can* fail, because it does the parsing itself, on this call, with this pattern. If a function that "just matches" hands you back an error, it is doing more than matching. ## What the extra work is Compiling means: parse the pattern text into a syntax tree (the `regexp/syntax` package), simplify it, and generate the program the matching engine runs. Matching a single short line, by contrast, walks that already-built program over the input. The compile step is the heavier of the two by a wide margin for typical patterns, and — crucially — it is *fixed per call* rather than proportional to how interesting the input is. Calling the wrapper once is negligible. Calling it once per log line, per row or per request multiplies a fixed setup cost by your throughput. You do not have to take that on faith. Two tools show it directly: - A CPU profile of the running program (`go tool pprof`) will show time inside `regexp/syntax` — parsing — rather than inside the matching engine. - A benchmark comparing the two shapes, run with `-benchmem`, will show both the time gap and the extra allocations the discarded compiled value causes. ## The idiomatic arrangement ```go var reqRe = regexp.MustCompile(`^GET /\S+`) func isGet(line string) bool { return reqRe.MatchString(line) } ``` The pattern is compiled once during package initialisation. Every call pays only for the match. Because the compiled value is immutable once built and its matching methods are safe to call from several goroutines at the same time, one package-level variable serves the whole program — you do not need a copy per caller or a lock around matching. If the pattern is only known at runtime, the same principle applies one level up: compile it once when you learn it — at start-up, at config load, or when a new distinct pattern first appears — and keep the `*regexp.Regexp` for as long as that pattern is in use. ## When the package-level function is fine It is not a trap, it is a convenience, and convenience is worth something: - a one-shot check in `main` or in a small CLI; - a single validation at start-up; - a test or an example where clarity beats a microsecond; - throwaway scripts. The rule is simply: **once is fine, per item is not.** ## A common half-fix A frequent partial answer in review is to hoist the *pattern string* into a constant and keep calling `regexp.MatchString(patternConst, line)`. That changes nothing: the string was never the cost, the compilation was. The thing that must be hoisted is the compiled `*regexp.Regexp`. The other half-fix is calling `regexp.Compile` at the top of each loop iteration and reusing it "within the iteration". That is still one compile per item; the reuse has to outlive the loop, not the iteration. ## What to say in an interview Name the mechanism (the wrapper compiles and discards), name the tell (it returns an error, the method does not), name the fix (compile once into a package-level var or a value built at start-up, call the method in the hot path), and name the evidence (a CPU profile showing parse frames, or a benchmark).
- How would you show the difference rather than assert it?Write two benchmarks over the same input — one calling `regexp.MatchString` per iteration, one calling the method on a package-level compiled value — and run them with `-benchmem`. The wrapper shows both a large time gap and allocations for the compiled value it discards. In a live service, a CPU profile showing `regexp/syntax` frames is the same evidence.
- A reviewer moves the pattern text into a package-level const and keeps calling regexp.MatchString with it. Has anything improved?No. The string literal was never the cost; the parse-and-compile on every call is. Hoisting the constant only removes a duplicated literal. The value that must outlive the loop is the compiled `*regexp.Regexp`.
- Is there any case where the package-level function is the right call?Yes — anywhere it runs once: a single start-up validation, a short CLI, a test, a throwaway script. The cost only matters when it is multiplied by throughput, and the wrapper reads more compactly when it is not.
It is the difference between preparing a SQL statement once and reusing it, and re-parsing the same statement for every row you touch. The text never changed; the parsing was the bill.
saying these in an interview costs you the question
- Assuming the package caches compiled patterns internally
- Hoisting the pattern string instead of the compiled Regexp
- Calling regexp.Compile once per loop iteration
- Thinking the wrapper's error reports a failed match
- Claiming the compiler prepares the pattern at build time