November 2011 Archives

Dist::Zilla, Pod::Weaver and bin

I use Dist::Zilla for managing my distributions. It's awesome and useful, although it lacks some bits of documentation every now and then; this lack is compensated in other ways, e.g. IRC.

I also use the PodWeaver plugin to automatically generate boilerplate POD stuff in the modules. Some time ago I needed to add some programs to a distribution of mine (which I also managed to forget at the moment, but this is another story), and this is where I got hit by all the voodoo.

The first program was actually a Perl program, consisting of a minimal script to call the appropriate run() method in one of the modules of the distro:

$ cat bin/perl-program 
#!/usr/bin/env perl
use prova;
prova->run();

This led Dist::Zilla to complain like this:

$ dzil build
[DZ] beginning to build prova
[DZ] guessing dist's main_module is lib/prova.pm
couldn't determine document name for bin/perl-program at ...

which isn't the best advice in the world (it actually complains about Pod::Weaver), but let's ignore it for the moment. After a bit of googling - or whatever, I actually don't remember - I found that there were basically two options:

  • put an explicit package declaration in the driver program, like this:

    $ cat bin/perl-program 
    #!/usr/bin/env perl
    package prova;
    use prova;
    prova->run();
    
  • put a comment with a PODNAME:

    $ cat bin/perl-program 
    #!/usr/bin/env perl
    # PODNAME: prova
    use prova;
    prova->run();
    

The latter seems slightly less dumb so I opted for it. I say slightly because IMHO the bottom line should be that the name is equal to the filename, but this is (again) another story. Yes, I know, it's open source and I can propose patches - did I say I'm not complaining?

Anyway, I then needed to add a shell script to the lot:

$ cat bin/script.sh 
#!/bin/bash
echo 'Hello, World!'

and again the error popped up:

$ dzil build
[DZ] beginning to build prova
[DZ] guessing dist's main_module is lib/prova.pm
[PodWeaver] [@Default/Name] couldn't find abstract in bin/perl-program
couldn't determine document name for bin/script.sh at ...

Now, I could use the PODNAME trick above:

$ cat bin/script.sh
#!/bin/bash
# PODNAME: script.sh
echo 'Hello, World!'

but it turns out - with little surprise - that the file is considered a Perl one, with the consequence that in the distribution it gets POD added to it:

#!/bin/bash
# PODNAME: script.sh
echo 'Hello, World!'

__END__
=pod

=head1 NAME

script.sh

=head1 VERSION

version 0.1.0

=head1 AUTHOR

Flavio Poletti <polettix@cpan.org>

=head1 COPYRIGHT AND LICENSE

blah blah blah...

=cut

The only thing is that - ehr... - bash does not like POD very much.

Google was not my friend in this case, but I found one in Dist::Zilla's IRC channel (which is #distzilla on irc.perl.org, by the way), which I think (hope) is Christopher J. Madsen, the original contributor of Dist::Zilla::Plugin::FileFinder::ByName.

He instructed me to use that module - which entered the core as of version 4.300003 - to obtain what I was after: keep the PodWeaver plugin working on Perl stuff, while ignoring shell stuff. But, more importantly, he adviced me about why.

Many plugins - including the PodWeaver one - rely upon the Finder role to do their work. This role is something that finds files to be used by the plugin; in the case of PodWeaver, the default is to take whatever module will be installed and whatever stuff is in the bin directory. OK, it can be a bit more complicated than this, but most of the times it's OK. This default is the same as if we configured the following in the dist.ini file:

[PodWeaver]
finder = :InstallModules
finder = :ExecFiles

It turns out that if we explicitly configure a finder all the defaults are wiped away, so - for example - if our bin directory contained shell scripts only we could be happy with this:

[PodWeaver]
finder = :InstallModules

In our case, anyway, this would disable PodWeaver for Perl programs as well, which is not acceptable. This is where FileFinder::ByName kicks in:

[FileFinder::ByName / BinNotShell]
dir = bin
skip = .*\.sh$

This plugin lets us create new finders. In the example above, we are creating a finder that finds stuff in the bin directory, skipping all files that end with .sh (note that skip sets a regular expression). At this point, we're ready for the configuration of the PodWeaver plugin:

[PodWeaver]
finder = :InstallModules
finder = BinNotShell

It's correct, the new finder does not want the initial colon. Now, when it's time for the Pod::Weaver plugin to find files, it will get all the modules that will be installed AND the files in the bin directory whose name does not end with .sh. Yay!

Exclusive Perl Archive Nook

I started working on epan, a (somewhat thin) wrapper around cpanminus to create a version of CPAN trimmed down to your needs for installing specific stuff.

This is what the cool guys probably call DPAN these days, but I found that the whole concept of DarkPAN revolves much around getting your private stuff into the "normal" Perl toolchain, while in this case I need to be able to easily install modules in machines that are out of Internet reach.

To start with an example, suppose you have to install Dancer and a couple of its plugins in a machine that - for good reasons - is not connected to the Internet. It's easy to get the distribution files for Dancer and the plugins... but what about the dependencies? It can easily become a nightmare, forcing you to go back and forth with new modules as soon as you discover the need to install them.

Thanks to cpanminus, this is quite easier these days: it can actually do what's needed with a single command:

      # on the machine connected to the Internet or to a minicpan
      $ cpanm -L xxx --scandeps --save-dists dists \
           Dancer Dancer::Plugin::FlashNote ...

which places all the modules in subdirectory dists (thanks to option --save-dists) with an arrangement similar to what you would expect from a CPAN mirror. Alas, on the target machine, you still have to make some work - e.g. you should collect the output from the invocation of cpanm above to figure out the order to use for installing the distribution files.

Additionally, the directory structure that is generated lacks a proper index file (located in modules/02package.details.txt.gz) so it would be difficult to use the normal toolchain.

epan aims at filling up the last mile to get the job done, providing you with a subdirectory that is ready for deployment, with all the bits in place to push automation as much as possible. So you can do this:

      # on the machine connected to the Internet or to a minicpan
      $ epan create Dancer Dancer::Plugin::FlashNote ...
      $ tar cvzf epan.tar.gz epan

transfer epan.tar.gz to the target machine and...

      # on the target machine
      $ tar xvzf epan.tar.gz
      $ cd epan
      $ ./install.sh

optionally providing an installation target directory:

      $ ./install.sh /path/to/local/perl

The epan directory that is generated should be compatible with other tools - in particular, the modules/02package.details.txt.gz file is generated, so all the toolchain (including cpan) should play nicely with it.

About Flavio Poletti

user-pic I blog about Perl.