Tom Wyant

Revision: that Cincinnati Perl Mongers found an error in the benchmark script used for this post. Match Anything Quickly - Revision 1 discusses their findings and links to a revised benchmark script. -- TRW 2022-09-02

Sometimes I want to filter a set of strings, but the details of the filter are not known beforehand. In particular, I may want a null filter, which simply accepts anything.

This looks like a job for a regular expression, but I can think of at least two implementations. One is to pass around regular expression objects. The second is to wrap a match (m//) in a subroutine reference, and pass that around. Given the use of regular expressions, there are a number of possibilities for a regular expression that matches any string.

I wondered whether one of the alternatives I was choosing among was faster than another, so I decided to Benchmark them. Both implementations applied the regular expression to a global variable. In practice this would probably be a localized $_, but my read of the Benchmark module says that it also localizes $_, but leaves it undef.

Note that the empty pattern is not benchmarked, because it is equivalent to the last successfully-matched pattern, if any. The sub { 1 } was included because if we're dealing in code references, the null filter simply needs to return a true value.

Here are the results, obtained with Perl 5.36.0, unthreaded. The script that generated them is on GitHub

Somewhat to my surprise, the subroutine-reference implementation was an order of magnitude faster than the regular-expression-reference implementation. I expected that, Regexps being first-class objects, it would be pretty much equivalent to m/ ... / wrapped in a subroutine -- maybe even a little faster.

A little messing around with perl -MO=Concise got me the following:

$ perl -MO=Concise -e '$_ =~ m/foo/;'
5  <@> leave[1 ref] vKP/REFC ->(end)
1     <0> enter v ->2
2     <;> nextstate(main 1 -e:1) v:{ ->3
4     </> match(/"foo"/) vKS ->5
-        <1> ex-rv2sv sK/1 ->4
3           <$> gvsv(*_) s ->4
-e syntax OK
$ perl -MO=Concise -e '$_ =~ qr/foo/;'
7  <@> leave[1 ref] vKP/REFC ->(end)
1     <0> enter v ->2
2     <;> nextstate(main 1 -e:1) v:{ ->3
6     </> match() vKS ->7
-        <1> ex-rv2sv sK/1 ->4
3           <$> gvsv(*_) s ->4
5        <|> regcomp(other->6) sK ->6
4           </> qr(/"foo"/) s ->5
-e syntax OK

The salient difference, to my eye, was the presence of the regcomp operator in the second case. perldoc-search on this led me eventually to perlreapi which says, in part,

"precomp" "prelen"

Used for optimisations. "precomp" holds a copy of the pattern that was compiled and "prelen" its length. When a new pattern is to be compiled (such as inside a loop) the internal "regcomp" operator checks if the last compiled "REGEXP"'s "precomp" and "prelen" are equivalent to the new one, and if so uses the old pattern instead of compiling a new one.

The relevant snippet from "Perl_pp_regcomp":

            if (!re || !re->precomp || re->prelen != (I32)len ||

                memNE(re->precomp, t, len))

            /* Compile a new pattern */

	

So I assume that the speed difference might be reduced if the filter was called in a tight enough loop. But if so, the Benchmark loop is not tight enough, and it's pretty tight. On the other hand, maybe the Benchmark loop is tight enough, and the extra time is spent determining that a recompilation is not needed. But it will take deeper knowledge of Perl internals than I possess to sort this out.