Re: What if we could drop archives into @INC?
An article written by brian d foy What if we could drop archives into @INC? got me thinking. Indeed, I've been burned myself by PAR, which is a great instrument, but I don't like that it does a huge unzip every time the application made with it starts. And I don't like even more so that some times PAR forgets to clean up these huge perl trees, eating space in my /tmp.
The problem why it is not that easy to override "use" and "require", is that these are semantic constructs, and are using perl's own file open/close operations, which neither are overridable even with *CORE::GLOBAL:: hack.
Nevertheless, after a couple of days learning about implementation of PerlIO layers and hacking them, the following module was born: File::Redirect. It's basically hacks into open() which is used by perl, and then emulates unix mount semantics.
Warning!!! The only problem is that it all works ONLY if perl was compiled with -DPERL_IMPLICIT_SYS, and that's AFAIU is by default windows perl/strawberry only. I believe one can build such perl on unix but I didn't try.
The simplest case would look such as:
use File::Redirect qw(mount); mount( 'Simple', { 'a' => 'Hello world!' }, 'simple:') or die; open F, '<a'; print <F>; close F;
which indeed does print 'Hello world'. But that's not all! -- we can do this too:
use File::Redirect qw(mount); mount( 'Simple', { '/Foo.pm' => 'package Foo; sub foo { 42 }; 1' }, 'simple:'); use lib qw(simple:); require Foo; print Foo::foo(), "\n";
and yes, perl is fooled into thinking that 'Foo.pm' is located at path 'simple:/Foo.pm'!
But even that is not all yet. The greatest part is that zip archives can be used, at last! Suppose we make a simple zip file, which looks like this:
$ unzip -l Foo-Bar-0.01.zip Archive: Foo-Bar-0.01.zip Length Date Time Name -------- ---- ---- ---- 0 02-25-12 07:46 Foo-Bar-0.01/ 0 02-25-12 07:47 Foo-Bar-0.01/lib/ 0 02-25-12 07:47 Foo-Bar-0.01/lib/Foo/ 43 02-25-12 07:47 Foo-Bar-0.01/lib/Foo/Bar.pm -------- ------- 43 4 files
And the only file of interest there is
$ unzip -p Foo-Bar-0.01.zip Foo-Bar-0.01/lib/Foo/Bar.pm package Foo::Bar; sub foo { 42 } 1;
The following code does it, without unpacking on disk:
use File::Redirect::lib ('Zip', 'Foo-Bar-0.01.zip', '/Foo-Bar-0.01/lib'); use Foo::Bar; print Foo::Bar::foo(), "\n";
So it work kind of like Java's jar files. Kind of, because of dynamic libraries, cannot be treated that way, but all other files can.
To sum it up, it's a pity that the module only works only on some perls, so possibly if there are arguments to use -DPERL_IMPLICIT_SYS on unix builds, we could do something about it?
There's a branch in the PAR SVN repo that loads modules from memory instead of unpacking on disk. But that doesn't work for shared objects. Presumably, with lots and lots of system-specific knowledge and tinkering, even that could be made to work, but I wasn't going to be the one to waste my time on it.
PS: The files remain in /tmp since PAR won't re-extract everything if the cache is warm.
@Steffen: that's interesting! is this the branch you're referring to?
http://svn.openfoundry.org/par/branches/fastio/
I can't find anything related there, sorry
Try: par/branches/fastio$ svn diff -c546
for an example commit that is related.
I think monkey-patching open at such a low level is not the most sensible way to approach this. It's bound to break some things.
Why not put a hook into @INC? It's a long documented interface that has far fewer issues than this.
I took 30 minutes and wrote a module to do this for you.
https://github.com/exodist/archlib
I also uploaded it to cpan, 'archlib' though it will take a couple hours to replicate out to mirrors.
Example:
use archlib 'path/to/archive';
use Module::In::Archive;
use Another::in::Archive;
Enjoy
@Chad
Thanks for the effort, I remember I tried that thing with @INC some years ago, when it wasn't established yet, and didn't work the way I hoped. It's great to know that it is solid now.