The first thing people lay their finger on when pperl comes up is always the same: "XS is not supported." It reads like a checkbox deficit - one feature, absent, evaluation over. This post takes that checkbox apart. It argues that "XS support" is not one feature but three almost entirely disjoint products sold under one name, that the delivery mechanism they share is the actual problem, and that the question "does it support XS?" is itself worth examining - because there are two very different reasons XS is everywhere in the Perl world, and only one of them is technical.

Let one thing be said before any of that: XS is genuinely useful for perl5, has been for three decades, and nothing below disputes it. Much of what makes CPAN valuable exists because XS existed. The point of this post is not that the investment was wrong. It is what happens when an investment starts answering questions it was never the answer to.

Why XS is everywhere

XS earned its place for two reasons, and they are worth keeping separate.

Reason one: there was no better option. Perl code needed speed, and perl5 has no JIT - so hot algorithms went to C (List::Util, JSON::XS, Text::CSV_XS, Class::XSAccessor). Perl code needed foreign libraries, and for most of perl5's life there was no FFI worth the name - so bindings went through compiled glue (every DBD, Net::SSLeay, XML::LibXML, GD, Cairo, SDL). Tool authors needed to see and hook the interpreter's workings, and there was never a designed API for that - but the raw structs were right there in the headers (Scalar::Util, PadWalker, Devel::Peek, Variable::Magic, B::*, Devel::NYTProf, Keyword::Simple). Three needs, one escape hatch.

Reason two: the investment. h2xs, ExtUtils::MakeMaker, Module::Build, the Alien::* build ecosystem, the CPAN testers matrix, "Extending and Embedding Perl", thirty years of tutorials and list archives and the collective muscle memory of a generation of CPAN authors. Effort spent became infrastructure. Infrastructure became ecosystem. Ecosystem became default.

Reason one is a technical argument, and technical arguments age with technology. Reason two is an economic argument, and it is the one the title is about.

The primer

The sunk cost fallacy, in one sentence: continuing to invest in a path because of what has already been spent on it, rather than because of where it leads. The textbook cure is a question: "knowing what you know now, would you choose this path today from scratch?"

Two clarifications, because the term gets thrown around as an insult and we do not mean it as one.

First, a sunk cost is not a fallacy. The decades in XS are real value, standing infrastructure, working code. For perl5, continuing with XS is not a fallacy either - it is the rational move, because reason one still holds there: on perl5 there is often still no better option, and abandoning working infrastructure with no replacement would just be waste.

The fallacy begins at a precise point: when the investment itself becomes the argument. When "does the new runtime support XS?" means, underneath, "is our sunk cost preserved?" - rather than "are the jobs XS was doing still getting done?" The first question evaluates the past. The second evaluates the runtime. This post answers the second, job by job. But first, the mechanism.

One name, three products

Look at what CPAN's XS modules actually do, and XS resolves into the three products already listed above: speed, foreign-library glue, and interpreter-internals access. Three products, almost no overlap in what they need - but one shared delivery mechanism, and that mechanism is the real problem: a shared object compiled against perl's headers, talking to the interpreter through its raw memory layout.

XS is not an API. It is an ABI - and an unstable one by design. XS modules must be rebuilt for every perl version, break on build-flag differences (threads, quadmath, DEBUGGING), and break at the source level on major releases. The current example is live: blead's reference-counted argument stack is forcing PERLRCSTACK adaptations across CPAN right now. And the dependency cuts both ways: perl5 itself cannot change its own internals freely, because half of CPAN has its hands inside them. The investment protects itself by making change expensive - for everyone, including perl5.

Adopting that contract would have forbidden pperl's existence. A runtime that reserves the right to JIT-compile your loops, run them in parallel, and reorganize its internals cannot promise that its memory looks like perl 5.42's forever. So the question we actually had to answer was never "why don't you support XS?" It was: which of the three products can be served without the ABI, and at what cost?

Product 1: Speed. Out-competed by design.

This one is the project thesis, so the answer is short. Three layers:

• Native modules. Over 60 modules of the core XS population are native Rust inside the binary: List::Util, Scalar::Util, Data::Dumper, Digest::*, Encode, Storable, Clone, Time::HiRes, POSIX, B, PadWalker, mro and more. They load with zero filesystem access and zero compilation, and they are tested byte-exact against real perl5 output.
• The JIT and the parallelizer. Pure-Perl code that stayed pure-Perl is not the slow path anymore. The whole reason Class::XSAccessor exists - perl's sub-call overhead - is attacked directly: compiled sub calls, compiled loops, native-function call-outs inside compiled loops.
• The interpreter itself. Readers of the 0.6.0 announcement will remember its caveat: the bare interpreter, JIT off, ran about 30% behind perl5. That caveat did not survive 0.6.1. The interpreter alone - JIT and parallelizer disabled - now runs the classic perlbench suite at an average of 108.6 against perl 5.42's 100 (higher is faster): on many rows pperl outruns perl5 before the JIT does anything at all.

Where a CPAN dist's speed layer is missing (List::MoreUtils::XS, say), the dist's own pure-Perl fallback runs - on a faster interpreter, with the JIT underneath. The "XS or slow" dichotomy that shaped CPAN packaging for twenty-five years does not exist here. Would anyone, building a speed layer today from scratch, choose per-version compiled shared objects over a JIT that ships in the runtime? That is the primer's question, applied.

Product 2: Foreign libraries. Substituted in kind, with named gaps.

The largest XS population by count, and the most important by river position. The correct primitive here was never "compile C glue per module" - it is FFI, and it is shipped:

• Peta::FFI calls into any system library with zero compilation: raw signatures (layer 0), curated bindings (layer 1), library discovery (layer 2).
• Peta::FFI::Cairo is the existence proof: a full-phenotype binding of a serious, stateful C API - streams, callbacks, glyphs, path iteration - pure declaration, no compiler anywhere. The original CPAN Cairo, last released in 2021, stops at cairo 1.16; this one binds 1.18.

What FFI costs relative to compiled XS glue, stated plainly - this is the part that is not black and white:

And one strategic move changes the economics of this entire front: implement the FFI::Platypus API natively. Platypus is CPAN's modern FFI standard, and a growing ecosystem of Alien::* + FFI::Platypus dists already binds C libraries without writing any XS. Platypus itself happens to be implemented in XS - but its user-facing API is small and stable. Ship that API natively on top of Peta::FFI, and that entire ecosystem installs and runs on pperl unmodified. CPAN compatibility bought at a single interface line, on our side of the ABI.

Note what happened to the sunk cost here: the Platypus ecosystem's investment is not discarded - it is preserved wholesale, because it was made against an interface rather than against memory layout. Investments made at interface lines survive runtime changes. Investments made against an ABI do not. That asymmetry is the whole lesson of this post, and CPAN's own FFI community had already internalized it before we arrived.

Product 3: Interpreter internals. The genuinely new work.

This is the product the checkbox framing always forgets. Kill XS and you kill Devel::Peek, Variable::Magic, every lint pragma that hooks the op checker, every profiler that timestamps ops. Right?

Two facts first.

A surprising amount is already served. The introspection layer CPAN reaches into C for is, in pperl, largely native already - because it is the sanctioned, semantic subset of internals access that perl5 itself maintains: Scalar::Util (weaken, readonly, blessed, refaddr, ...), Sub::Util, mro, PadWalker, and B - complete enough that B::Deparse and B::Concise work end-to-end.

PadWalker deserves a close look, because it is the pattern. PadWalker is as internals-hungry as modules get: it walks the context stack, pokes through pads and closures, resolves variable names. The pperl version is a native, function-by-function faithful port of PadWalker 2.5: all six public functions (peekmy, peekour, peeksub, setclosedover, closedover, var_name), audited line-by-line against the C, byte-exact against real perl5 across 184 comparisons - including UTF-8 variable names and the exact error message spellings. Exactly one export was not carried over: _upcontext, an undocumented function whose return value is a raw interpreter context pointer cast to an integer. That is not an accident of effort. It is the design line: we reproduce what internals modules do, never what they touch. A raw pointer is the ABI trying to climb back in through the window; the moment we hand one out, our internals are frozen again.

Why the rest of this product is C on perl5 - and would not be here. A magic vtable, an op-check hook, a profiler tap: none of these need C's expressiveness. They are C on perl5 for two reasons: perl5 cannot afford a Perl-level callback in those paths (a coderef call per variable read would be ruinous at perl5's call cost), and perl5 never designed a semantic API for them - the raw structs were simply there. Both reasons are instances of reason one from the top of this post: no better option, at the time, on that runtime. pperl's economics are different. Compiled sub calls are cheap, and a callback that becomes hot is itself a JIT candidate. The cost argument for C evaporates; what remains is to design the API perl5 never did.

That design has a name: Peta::XS (working title) - the interpreter's inner workings exposed to pure Perl as a versioned, semantic API. Never memory layout. Tiered:

• Tier A - inspection, read-only, always safe. SV anatomy (flags, refcount, value slots, magic list), full op-tree walking (B is the genotype; it needs completeness, not invention), stash and glob topology. Phenotypes: Devel::Peek, Devel::Refcount, Package::Stash, coverage tooling.
• Tier B - structured mutation. Perl-level magic: attach get/set/free coderefs to a variable (Variable::Magic's wizard API - the single highest-leverage internals phenotype on CPAN). Compile-phase op-check hooks with coderef callbacks (autovivification, indirect). A keyword plugin receiving source text (Keyword::Simple, the gateway to the syntax-extension dists). Profiler and tracer taps (enter/leave-sub, per-statement) - the sanctioned way to build Devel::NYTProf-class tools, and pperl's own profiler should be this tier's first consumer.
• Tier C - explicitly rejected. Raw peek and poke, addresses, struct offsets, anything whose contract is "the memory looks like perl 5.42's". Each of these has a Tier A/B substitute for its legitimate uses; what they do not get is the ABI back.

Where a phenotype matches a famous CPAN module, that module's interface ships natively on top - Variable::Magic, Package::Stash, Keyword::Simple as thin skins over the versioned core, exactly like the Platypus plan on the FFI front. Compatibility at interface lines we choose, not at the memory layout.

What stays lost

No substitution story covers everything, so here is what this one does not cover:

• cpan install Foo::XS will not work. Ever. An XS dist runs if its pure-Perl fallback exists, if a native or FFI equivalent ships, or if its interface skin exists. The mitigation is coverage of the river head - the same 80/20 that made 60+ native modules viable - not universality.
• Exotic C-side embedding. Hosting the interpreter inside a C program (mod_perl-class), C code linking libperl, PerlIO layers written in C. Out of scope until a real user appears.
• Bug-compatible internals behavior. Dists whose test suites assert perl's private quirks will diverge. Acceptable by definition.

The reframe

"XS not supported" is true the way "this electric car does not support gasoline" is true. XS was three products. The first - speed - we do not substitute, we out-compete; that is the reason pperl exists. The second - foreign libraries - is served by the correct primitive, FFI, with named gaps and funded work items, and a single-point compatibility play in Platypus. The third - internals access - is the genuinely novel work: the semantic API perl5 never designed, with the famous CPAN interfaces as skins, and PadWalker already demonstrates the standard: everything the module does, byte-exact; nothing the module touched.

And the sunk cost? It was never wasted. Thirty years of XS built much of the CPAN we all stand on, and on perl5 it remains the rational choice, because there XS still is the best option available. A sunk cost only turns into a fallacy at the moment it is allowed to choose the future. So when a new runtime comes along, the question that evaluates it is not "is the old investment preserved?" - it is "are the three jobs still getting done, and done better?"

One checkbox, three roadmaps. Two of them mostly shipped.

• Richard C. Jelinek, PetaMem s.r.o.