Auto(Split|Loader) in a modern Perl world
As a few of my previous posts have implied, I am attempting to reinvigorate the Zoidberg Perl shell. Much of the work of getting it back to a functional state has already been done at my GitHub repo. I have a bigger post coming on why this is cool and even another with some examples, but for now I have a question:
Is an AutoSplit/AutoLoader mechanism helpful on modern hardware? I mean Moose/MOP (and many other projects) are huge and doesn’t use it. In fact it seems that very few modules depend on it.
Now, I understand that AutoLoader has some cool uses for causing subs to spring into existance. In the context of Zoidberg, though, I only care about its use to defer loading infrequently used subs.
To test some things, I created a branch in which I naively removed these bits, and lo and behold, with only one missing my (see AutoLoader Considered Harmful), the tests all pass and a quick run seems fine.
So I fairly call the question: should I leave AutoLoader in, or pull it?
As I delve into the deeper Perl magic I like to share what I can.
I wouldn’t choose to use it unless you have a huge lot of Perl (tens of thousands of lines) that you’d rather not fully parse on startup. You pay for that in more files to read from eventually. Doesn’t mean it’s worth to explicitly migrate away if it works for you, though.
OK, it works – but does it make any difference? If it doesn’t, what’s the point of having more moving parts in a program?
Personally, I would like for AutoLoader to disappear from all core modules. Not the use of deferred compilation – but the use of AutoLoader. (Isn’t it a clear indictment when the one core module which really needs deferred compilation, CGI.pm, has always used a custom scheme to implement that?)
Maybe even from the toolchain – maybe it can be relegated to legacy code remaindered on CPAN. But that would be a far more ambitious goal.
If you want a canned solution, I suggest taking a look at SelfLoader. It gives you only one-shot deferred compilation, but it keeps the incremental cost to a minimum.
Otherwise, I suggest you roll your own. It is not difficult to do with
AUTOLOADandeval.Sorry I think I need to make myself clear. Zoidberg already uses AutoLoader, and, as its new maintainer, I am pondering removing it. Obviously I will need to make sure that I don’t introduce any new bugs, however, other than that, is there any reason not to remove AutoLoader from an existent project?
Ah. As you will imagine after my previous comment – I am all for that. :-)
I don’t see any reason. It is supposed to be drop-in. So it should just as easily drop-out. I have not heard of any gotchas.
Well, why was AutoLoader used in the first place? It was used because this is a shell, and you want really fast startups for shells, especially if they are called from other programs to execute something. I say you should run some benchmarks to see what happens if you simply remove it vs if you use SelfLoader.
To be clear, Chapter 29 of the soon-to-be-replaced Camel Book tells me that “system” does not use the user’s shell to process the arguments, it uses /bin/sh (footnote, pg 812). So you don’t need to worry about Perl using this shell to handle “system” or “exec” calls. But speed should still be a concern.
For a user’s default shell, you will want something that loads fast. AutoLoader may actually be the right solution here, for once. :-)
Do not confuse compilation deferral (a general concept) with AutoLoader (one possible implementation of the concept). AutoLoader is not necessarily the right tool even if deferred compilation is worthwhile for Zoidberg (which itself is not something I’d presume merely because it is a shell).
@Aristotle - good point, but I didn’t make that confusion. My point was that the fewer bytes Perl has to read from disk at startup, the faster it can load the program. That could be a small or a big win, depending on the size of the codebase (which I haven’t closely examied).
Perhaps the more important point (against my original), Joel is the new maintainer of a module, which means he’ll be adding features. He’ll need all the help from Perl::Critic et al. as he can get. In the (very) rare situation that it’s useful, AutoLoader is a finishing tool for speed. It would probably be better to strip it out until new features have been added and debugged, and then only added back in if it gives a big performance win.
I’m even less convinced by that: what difference does it make to load 50kb or 500kb when contemporary spinning-rust hard disks read well over 100MB/sec? (SSDs are on another plane entirely of course.) Loading lots of little files OTOH can definitely be a perceptible slow-down if it requires hitting the disk at a bad time when the system is already loaded. If anything, you are arguing for SelfLoader.
As a follow-up, Zoidberg 0.98 has been released, and it does not use AutoLoader. The vestiges remain as comments, until I see if there are any lingering problems (not likely). By the next release, I expect to remove those comments.