PetaMem

Two weeks ago we announced PetaPerl 0.6.0. Today, 0.6.8 goes live at https://perl.petamem.com. Again, the version has no hidden meaning, but if you insist, you can call this the "дурак Go Home" edition. The headlines: pperl has now perl 5.44 as compatibility target because it follows the current stable perl5. The benchmark suite that produces our numbers is released, so you can see what the numbers are on your machine. DBI works, with native database drivers - connect to your PostgreSQL or MySQL database with an unmodified use DBI; script and no driver installation. And there is now a daemon that lets repeated runs of a script skip compilation entirely: each run forks an already-compiled image and goes straight to execution.

5.44: the target moves, so do we

Perl 5.44 is out, and pperl's compatibility target moved with it: runtime behaviour, error message spellings, and the documentation set are now measured against 5.44, not 5.42.

That is the policy: the target is the current stable perl5, not a fixed snapshot. Perl 5 is a moving target, and pperl tracks it - 5.42 was the target while it was current, 5.44 is now, 5.46 will be when it arrives.

Operationally nothing about the method changes: the test harness still runs everything against both implementations and compares output byte for byte - the reference binary is now perl 5.44.

Speed, and the suite that measures it

0.6.8 is faster than 0.6.0 - across the interpreter, the JIT, and the parallelizer. The 0.6.0 announcement already warned that its own numbers "age fast"; they did, in our favour, and they are still aging. So this post does the consistent thing and does not print a benchmark table at all. Instead, the release ships the benchmark suite itself: https://perl.petamem.com/downloads/pperl-bench.tar.gz. The suite measures perl5 and pperl side by side on your machine, on your workloads if you extend it - three pperl configurations per row (interpreter only, JIT single-core, JIT+Rayon), with the measurement discipline built in: in-program rate timing, a system-load gate so a busy machine cannot flatter either side, and no baseline-subtraction games.

What grew since 0.6.0, in words rather than decimals: the JIT stopped being a benchmark-shape compiler and started being a code compiler - loops inside subs, nested and C-style loops, hash reads and writes with computed keys, closures captured by compiled callees, map and grep, two-dimensional grid access through reference chains. The test case we watch is not a microbenchmark anymore; it is real module code compiling as whole loop nests. The parallelizer learned shared read-only arrays and grid workloads, gained a measured cost model, and - the rule we hold it to - bails out rather than ever running slower than the interpreter would have. Run the suite and see where that lands on your hardware.

DBI, with drivers

Databases have been the single most-requested piece of the CPAN river since the first announcement. 0.6.8 ships a DBI 1.651 compatible module embedded in the binary, plus native driver implementations: DBD::Pg (PostgreSQL), DBD::mysql (MySQL and MariaDB), and - as an experimental preview - DBD::Sybase, speaking TDS to Sybase ASE and Microsoft SQL Server.

The user side of it, in full:

use DBI;
my $dbh = DBI->connect("dbi:Pg:dbname=inventory;host=db.example.com",
                       $user, $pass);
my $rows = $dbh->selectall_arrayref("SELECT id, name FROM widgets");

Run that with pperl and it connects; there is no second step: no DBD::Pg build against server headers, no C compiler, no XS toolchain, no version-matched driver reinstall after a perl upgrade. The drivers are pure Perl, embedded in the runtime, and talk to the same client libraries the CPAN drivers bind - libpq, the MySQL/MariaDB client library, FreeTDS's ct-lib - through Peta::FFI. The client library itself is the only thing expected on the system.

Where the drivers stand: DBD::Pg and DBD::mysql are verified end-to-end against live servers - the test suite spins up its own PostgreSQL and MariaDB instances and runs the full CRUD path against them. DBD::Sybase ships as experimental, and the label is precise. Its client half - connection machinery, login properties, host/port routing, error and diagnostic reporting - is verified against real ct-lib behaviour. Its server-conversation half - the login handshake and the result pipeline - is written faithfully against the ct-lib API but has not yet talked to a live server. The test suite is armed for the day one is reachable; until that run has happened, it stays experimental.

XS - again

Which brings up a sentence that has appeared in every post of ours so far and is due a correction: "pperl does not support XS." That sentence is technically right and factually wrong. It misleads most readers - including, maybe especially, readers with real XS experience - because it lets "XS" pass as one thing, which it isn't. It is three almost unrelated products sold under one name - speed, foreign-library glue, and interpreter-internals access - held together only by their delivery mechanism: a shared object with its hands in the interpreter's raw memory. The mechanism is what pperl does not support, and will not. The products are a different story: speed is the project thesis, foreign-library glue is what the DBI section above shows doing consequential work, and the internals API has its first version in this release. The precise statement is: no ABI - and two of XS's three products delivered, with the third under construction.

The daemon: fork is our multiplicity

Every interpreter invocation normally pays the full start-up bill, and for real programs the expensive line on that bill is not process creation - it is compiling the script and every module it uses, again, on every run. 0.6.8 adds a daemon that lets repeated invocations of a script skip that line entirely: opt a script in, and the daemon keeps a fully compiled resident image of it - a template - so every further run is a fork of that image that goes straight to execution. Parsing, module loading, and BEGIN work happen once, when the template is built; after that, each run executes only the run phase.

The design leans on an architectural property of the runtime. The pperl interpreter is deliberately non-threaded and lives in a static image - which rules out perl5's ithreads-style in-process cloning, and that is not a loss but the enabler: fork() becomes the multiplicity mechanism. Children share all pages with the daemon copy-on-write; nothing is re-linked, re-constructed, or re-compiled.

Measured: a module-heavy script that cold-starts in 24.5 ms comes back from its template in 3.3 ms - 7.5x, and the win grows with the weight of the modules, because the win is the compile time. A trivial script that loads nothing gains roughly nothing - the saving and the daemon's own overhead cancel - and never meaningfully loses.

Because compile-time side effects are frozen into the image, templates are strictly opt-in, per invocation, never the default - the same preload contract that application servers established a generation ago: open your connections at run time, read your configuration at run time. Everything that must be fresh per run is fresh per run: @ARGV, %ENV, working directory, standard streams, $$, exit codes and signals, __DATA__. And whenever the daemon cannot reproduce an invocation exactly - different flags, no daemon running, an environment variable that shapes interpreter start-up - pperl silently falls back to a normal cold run. Correctness never depends on the daemon; only speed does.

The details - commands, the opt-in contract in full, cache behaviour, security model - are in the documentation's performance guide.

-I, use lib, PERL5LIB: repaired

An apology is in order here. In previous versions, pperl's module search path was - not to put too fine a point on it - obnoxious: the binary carried one machine's directory layout baked in as string literals. On that machine everything worked; on a Debian or Fedora layout use strict alone could kill a script, and -I, use lib, and PERL5LIB interacted with the baked list in ways no perl5 experience would predict. 0.6.8 replaces that with a real @INC construction:

• The standard Arch, Debian, and Fedora pure-perl tree locations are probed at startup, not assumed - and versioned trees are ordered by nearness to pperl's own version, with a host perl of a different version used gracefully as a last resort rather than rejected.
• -I, use lib, and PERL5LIB now behave the way your perl5 reflexes expect: first hit wins, in the order you gave.
• Architecture directories are skipped entirely - pperl loads no .so modules - which is what lets library trees built by a real perl (local::lib, Carton) work unmodified.
• The interpreter-coupled core modules (strict, warnings, feature, lib, builtin, ...) are embedded in the binary, so use strict or a -E one-liner no longer depends on any host perl tree existing at all. Built-in modules keep absolute priority; no on-disk .pm can shadow them.
• pperl -V shows the @INC that was actually constructed.

The full rules, including the pperl-private overlay locations for parallel installations, are in the documentation under "differences from upstream".

Documentation, and the other tidbits

The documentation kept its own pace. The reference set now tracks 5.44 alongside the runtime. The "Definitive Guide to Graphics with Perl" is translated into all 41 non-English locales. And a set of smaller things landed that individually never earn a post, among them -MO=Deparse and -MO=Concise working end-to-end on the native B.

Politics & Outlook

A short word on the edition name. PetaMem is a Czech company. In 1968, the walls of Prague said «Иван, иди домой!» - Ivan, go home - to the tanks that had arrived overnight. The sentiment has not aged a day; we merely updated the addressee. That is all this post has to say about it.

Now the Perl politics, restated. pperl follows perl5; it does not lead, and it has no ambition to. The language is defined by perl5 and the people who maintain it - the "5.44" section above is that policy in operation, and it is permanent. There is no pperl dialect coming, no "improved" syntax, no embrace-and-extend. What pperl brings is of a different kind: speed, ease of deployment, and possibly reach - carrying Perl into places it has so far been barred from. It is meant to add to the Perl ecosystem, not to take from it.

As for the outlook: we have barely started. Being free of some of perl5's constraints lets us explore paths that are frankly experimental - things we cannot guarantee and will not promise, because we do not yet know whether they will work out. One of them is under active work: Project Longshot, offloading Perl code to the GPU - the name says what we think of the odds, and we are working on it anyway. Further down the line: Project Liliput, programming your favourite microcontroller in Perl. There are more. No dates, no promises; numbers when there are numbers - stay tuned.

• Richard C. Jelinek, PetaMem s.r.o.