PetaMem

Over twenty-five years ago, when PetaMem was still in its infancy but the use of Perl for its NLP/NLU systems was already a given (Natural Language Processing / Natural Language Understanding - we did not dare to call it AI back then), slightly before the 5.8 era and in a still-young SMP environment, I envisioned a perl that would autoparallelize. That is: dispatch your code onto the available CPUs, automatically. The idea was not mine. I had seen it sometime in the nineties at the CeBIT computer fair, at the SGI booth, where their C compiler was doing exactly that.

The vision evolved into a wish, and the wish, refined over the years, into a resolution: should PetaMem ever have the resources to fund the R&D team this would need - realistically at least twenty people, or five Jonathan Worthingtons - we would do it. 2025 was a remarkable year for us technology-wise, and when Christmas came, the decision was made: requisition said twenty … or fifty “people”, and build an autoparallelizing perl with a JIT.

PetaPerl was born.

The story so far

The project made its public debut at the German Perl Workshop 2026 in Berlin this March, and has lived on its own website at https://perl.petamem.com since. It has been mentioned here and there, discussed on Reddit, found its way to Hacker News - so it is not exactly unknown. But it was never officially announced. This is it, now.

Readers of this blog have already met the project sideways:

• PDL in Rust - A Native Reimplementation of the Perl Data Language
• Rust-PDL Part Two
• The Perl Documentation - Rewritten

What PetaPerl is

pperl - the binary name can stand for peta-perl or parallel-perl, take your pick - is a reimplementation of Perl 5 in Rust. Not a port of the C interpreter, not a transpiler for Perl code, not a “Perl-like language”: a runtime that aims to run your existing Perl 5 code unchanged, with 5.42 as the compatibility target. Correctness is defined operationally: the test harness runs every test against both perl5 and pperl and compares output byte for byte.

On top of that baseline sit the two things the dream was about:

• A JIT compiler, built on Cranelift, that compiles hot loops and hot recursive subs to native machine code.
• Automatic parallelization, built on Rayon, that spreads qualifying loops across the available CPU cores.

Both are automatic. No annotations, no pragmas, no use threads-style opt-in, no code changes. You run pperl script.pl and your loops get compiled, and where provably safe, parallelized. The SGI booth, thirty years later, for Perl.

There is one deliberate architectural departure: pperl does not support XS. XS ties modules to the C interpreter’s internals, which makes JIT compilation across module boundaries impossible and thread safety a minefield. Instead, over 60 core-ish modules that are XS in the original ship as native Rust reimplementations compiled into the binary, each validated against the original’s test suite plus our own. The two Rust-PDL posts linked above document what that looks like for the most ambitious case - and the native reimplementation of Cairo.pm certainly qualifies as the second most ambitious: the full binding surface of CPAN’s Cairo and then some - the original, last released in 2021, stops at cairo 1.16, while ours binds 1.18 - with no XS and no C glue. It arrived alongside the “Definitive Guide to Graphics with Perl” described below; sidelines like these happen while the main work is elsewhere.

Where we are today: 0.6.0

The version number carries no hidden message - it is plain semver, and 0.6.0 is the next logical step: a more mature code base, more JIT coverage, more Rayon, more performance.

That last word deserves numbers rather than adjectives. Since the GPW presentation, the JIT has grown from numeric loops into a compiler that recognises whole constructs: string operations, array and hash access, map/grep/sort/join/split, named sub calls including recursion, and regex match and substitution running the real regex engine from compiled code.

The benchmark suite measures pperl - JIT and parallelization on, as they are by default - against perl5 over the 42 core-language micro-benchmark families of the project suite: expressions, sub calls, loops, string functions, and regexes. The geometric mean over the whole set is 6.8x in pperl’s favour. A representative excerpt, in operations per second:

The spread is wide, and stating it honestly is the point. Arithmetic, numeric comparisons, and sub calls sit between 20x and nearly 500x - those are the shapes the JIT compiles outright. String functions, substr/index/sprintf, and substitutions typically land between 2.5x and 10x. Hash access, sorting, and regex matching are more modest at 1.2x to 2x. On top of all this sits the parallelizer: a Mandelbrot render distributed across cores comes out around 400x faster. If you force us to a one-line summary: depending on how much of your hot code the compiler can claim, expect somewhere between 2.5x and two orders of magnitude.

The top rows deserve one more sentence, because a skeptical reader should ask what a three-digit speedup on a scalar assignment can possibly mean. That benchmark body is loop-invariant - a constant assignment in a loop - and compiled it becomes a register move inside a native counting loop, parallelized across eight cores: 17.0 Gop/s is about two CPU cycles per iteration per core, which is simply the hardware speed of an increment-compare-branch loop. perl5 meanwhile interprets one op dispatch per iteration and reaches 86.6 Mop/s. Nothing is cheated - the final value is written back through the real setters, and the output stays byte-exact. The honest reading of such a row is not “pperl assigns scalars 200x faster”; it is “the compiler removed the interpretation, and what remains is the hardware”. Which is, after all, the point of a JIT.

One more thing about these numbers: they are a snapshot of a work in progress with a lot of headroom left, and they age fast. A week before this post, the same suite read roughly ten times worse. And while this post was being drafted, the table above contained a row reading 0.6x - numeric comparison, the one family where perl5 still won - which the draft duly named as the honest exception. We tasked the JIT instance - one of the fifty “people” from the opening - to have a look at the outlier. Verdict: the old number came from a stale binary on which the JIT never fired. Measured on the current build, the row reads 486x. We could dance on every decimal here; next week’s build will have moved the floor. Take the table as a bearing, not a boundary.

And the honest caveat, because numbers without one are advertising: the bare interpreter, with JIT and Rayon disabled, is currently about 30% slower than the original. That gap is real, we measure it continuously, and the target for the bare interpreter remains parity with perl5 - the JIT is not accepted as an excuse for it.

Two design principles make the fast numbers trustworthy rather than reckless:

1. Exactness. Compiled code produces byte-identical output to the interpreter, always. The harness compares pperl’s output byte-for-byte against perl5 on every test, with the JIT and parallelizer on. Floating-point loops evaluate in Perl’s own expression order; parallel reductions are gated on provable bit-exactness.
2. Decline, never guess. Anything the compiler cannot prove it can reproduce exactly, it declines, and the loop runs interpreted at normal speed. There is no mode in which compiled code is “probably right”.

Meanwhile, the documentation

The documentation effort described in the earlier post has not stood still either. The documentation is now available in 42 locales. And it is not only translation catching up with existing text: genuinely new content has landed, including a “Coming from …” series for readers arriving at Perl from Python, Ruby, PHP, Lua, Tcl, awk, sed, or the shell, and a “Definitive Guide to Graphics with Perl” that runs from raster foundations through Cairo and GUI toolkits to two capstone projects.

Try it

Everything described here is live at https://perl.petamem.com, where you can download the 0.6.0 binary and read the documentation. The architecture guide has a chapter on the JIT if you want the machinery in detail, including how to watch it work on your own code (--jit-stats, PPERL_JIT_LOG).

Two things belong in every honest announcement of this kind.

Do not take our word for any number in this post. Everyone is encouraged to run independent performance measurements and to review correctness on their own code. Two implementations checking each other is the strongest correctness argument there is; a community checking both is stronger still.

Not for production use. We have spent considerable effort making pperl behave exactly like the original, but we cannot guarantee that yet. Ourselves, we trust it today for small scripts, and for scripts that have been tested in a sandbox and proven to work - pperl is deterministic, so what has been proven to work keeps working. It does not yet run the large frameworks and 100k+ LOC codebases.

No dates for what comes next; we will post numbers when there are numbers to post. But the dream has a version number now.

• Richard C. Jelinek, PetaMem s.r.o.