I measured the size of some big modules, with B::Stats and B::C, statically compiled.

Interestingly Module::Build is much harder to compile - you'd need 6GB RAM at least -
and much bigger in the end-result than Moose, which compiles/compresses really fine.

Modules

Module::Build

perl -e'use Module::Build: print q(k)'

I catch myself always saying, "No, not yet stable enough." I cannot release it, even though it passes all tests. Which you can interpret that the tests suck. Not enough coverage, bad testcases, ...

Well, that is always the case. You can never have enough tests. Problem is that in my case, the compiler, testing costs a lot of time. LOT of time! I usually spend a week to do the final release testing, but more often it lasts several weeks, because one round of test results influence the decisions of TODO, SKIP and mandatory PASSing tests, and then I'll redo the tests. On all versions, with all platforms. You could rely on cpantesters to do that for you, but it is better to do the most common combinations by your own. That's why I use perlall with a few hundred perls.

But with passing tests you always have your definition of TODO tests. A todo test is always a sign of instability. "Sometimes it works, but not always". Or "It used to work, but it is not so important if it fails". Or "It used to fail, but somehow it looks like I fixed it now. But I'm not so sure".

But I came in the last years to a completely different definition of "stable". I call my app stable,

1. if the testsuite passes, AND

2. if small innocent changes to the source create expected results.

That means after a passing testsuite I always play around with the code a bit, doing minor improvements, or testing new features, and only if the results come out as expected I will call it stable enough. Only then I can trust my code.

I was often bitten by the "Action at a distance" anti pattern. Very often minor changes caused something completely unrelated to fail. E.g. loading another module, suddenly broke something which always worked, for no apparent reason.

E.g. a concrete perl example: PL_regex_padav is only relevant for threaded perls, holding the REGEXP bodies of stored qr// SV's. 5.8.1, 5.10 and then 5.16 changed the internal implementation of the PL_regex_padav offsets. 5.16 failed in the C compiler, but the same fix to the Bytecode compiler which looks sane fixed the Bytecode problems. The Bytecode compiler is much simplier than the C compiler and big implementation changes cause always synchronous changes in both. If you fix it in the Bytecode compiler you'll have to do the analogue in C. But in C suddenly all fell apart. The good thing, only in threaded perls > 5.15, so the errors are expected and isolated. Just the fix is not right yet.

Fixing compiled C code is always easy by debugging into it with gdb, one session native and one parallel session compiled, find a proper breakpoint and then compare the state. The reason why C failed could be related to something completely different. In C the PL_body_arenas were empty, and when it was initialized by sideeffect in the added fix (analog to the Bytecode fix), the whole PL_regex_padav array fell apart. gdb hw whatchpoints to check who is writing to it did not work.

Okay, the fix was just not good enough you could say. It works in Bytecode by accident but not in the general case. But this does not sound right to my experience. Something else not yet understood is going on. So I'm calling it instable.

Or if one fix in a 5.10-5.14 non-threaded case, causes changes for threaded code for no apparent reason.

Executive summary: With big complicated apps even after a passing testsuite and passing Q&A, either let Q&A play with it for some time or better play with it by yourself and see how it behaves. One additional week always plays well.

The latest B::C package on CPAN 1.42 works stable for almost all perls until 5.14, but so far did not work good enough for the upcoming 5.16 release.

I couldn't even pinpoint to a specific perl change which caused the problems. I know that hashes need a different initialization now. Empty hashes need to declare

HvTOTALKEYS(hv) = 0

after creation, and readonly hashes must be set readonly after they were created.

DynaLoader and %INC handling is much stricter now with 5.16.

And then there is the PMOP pmoffset IV hack. We have now first-class REGEXP SVs, but not really. With threads we still store the regexp body in PL_regex_padav, not as normal body in an arena, and the parser stores the latest pmoffset (I think) behind the PV in PL_regex_pad[0]. Thing is, the compiler does not use head and body arenas yet, it uses static arrays. But when the first body arena for some dynamic SV is initialized, the PL_regex_padav is reset.

All these changes forced me to rewrite the tricky parts of the compiler, the recursive walker to detect used packages and objects.

In my desperation I added checkers to detect the name of objects for method calls, to check bless for used object names, to associate blessed and new scalars with a method call, and to check bareword require for dynamically included packages. The ISA search is now recursive and tries a lof of candidates to find unknown objects for method calls, with proper AUTOLOAD and UNIVERSAL fallback.

This increased the compile time dramatically, but it must be correct and should include all possible used packages. Or if not, at least the %INC hash must be correct to allow dynamically added packages to be loaded properly at run-time. Possibly with DynaLoader.

Still, 5.16 did not pass the most tricky tests and production code, which all worked fine with 5.14 back in November.

Lately I fought deep recursion troubles when compiling some recursive functions in tricky package inclusion scenarios. Moose and recursive Pod::Simple functions mainly. Either the compiler missed some functions, stored some packages only halfway, like Encode (UTF8 did not work), or the compiler went into deep recursion issues in the walker. So I went the old-school way and removed too recursive functions from the walker.

First I disabled recursing into op->first, because the B function walkoptree already steps into op->first.

Second I disabled walking into newly found packages immediately and just mark them as new. The main walker loops now over all packages again and again until the list of newly found packages in each loop is empty. This adds costly compiler passes, and certainly increases the size of the produced binaries, but better load such code at once compiled, then defer it to later.

https://github.com/rurban/perl-compiler/compare/recurse