Perl Dependency Checking
I'm working on a few projects right now, most notably one that helps me create a CPAN distribution so that I can create a Perl Lambda in the AWS environment. This has led me to some yak shaving exercises, most notably investigating how to check for Perl dependencies.
Without getting too far into the weeds on Perl Lambdas (that's another blog post in the writing), suffice it to say I need to vendor Perl modules and deploy them in the Lambda environment. I briefly looked at carton and that may solve the problem neatly, but my early dive indicated to me that another path might be a more direct shot on goal and produce a cleaner Lambda deployment methodology.
Back to the issue at hand...specifically this blog is going to discuss Perl dependency checking using these tools:
scandeps.pl Devel::Modlist /usr/lib/rpm/perl.req
I'm sure I'm over complicating things and experienced CPAN authors will most likely point to the current set of methods available for both dependency checking and building CPAN distributions. Nevertheless my confusion given the various methods available and the lack of a single authoritative voice compelled me to create something simple (at least from my toolchain's perspective) for creating CPAN distributions from uncomplicated projects.
Of course if it's a simple distribution, why not just create a Makefile.PL as part of your project and be done with? Sure, that works - but my needs and desire for more automation have gotten the best of me over the holidays - hence - make-cpan-dist
This is not an ad for that tool - I doubt anyone is interested, however, I wanted to blog a bit about dependency checking in the hopes that some of you that read these blogs might illuminate the broad, dark corners of my ignorance.
Some background. Where I work we have packaged Perl applications modules using the Redhat Package Manager (rpm) as we are very coupled to the RedHat/CentOS environment. We have persevered with the system Perl for lo these many years with no major issues.
Aside: Truth be told, it worked for us but we are now moving a lot of our new development to Python for many of the reasons that have been hashed out in many other blogs and internet forums. Personally, I enjoy writing Perl and although I direct development efforts, our needs have dictated rethinking some of our development methods including what language and frameworks to use. We'll still have Perl around for some time and I will continue writing Perl code until I find it no longer meets my needs.
The Redhat Package Manager includes a script (/usr/lib/rpm/perl.req) that manages to do a fairly decent job of teasing out the direct Perl dependencies of your module. One of its advantages is that it seems to semantically check for dependencies in a more complete manner than some other tools. For example take this simple Perl script:
$ echo "use parent qw/Foo::Bar/;" > foo.plThe rpm tool reports Foo::Bar as a dependency, whereas scandeps.pl does not.
$ scandeps.pl foo.pl No modules found!
Now let's try using the -Rc options (don't recurse, compile).
$ scandeps.pl -Rc foo.pl Can't locate Foo/Bar.pm in @INC (you may need to install the Foo::Bar module) (@INC contains: /usr/lib/bedrock/perl5 /opt/perl-5.28.1/lib/site_perl/5.28.1/x86_64-linux /opt/perl- 5.28.1/lib/site_perl/5.28.1 /opt/perl-5.28.1/lib/5.28.1/x86_64-linux /opt/perl- 5.28.1/lib/5.28.1) at /opt/perl-5.28.1/lib/5.28.1/parent.pm line 16. BEGIN failed--compilation aborted at foo.pl line 5. SYSTEM ERROR in compiling foo.pl: 512 at /opt/perl- 5.28.1/lib/site_perl/5.28.1/Module/ScanDeps.pm line 1448.
Let's use /usr/lib/rpm/perl.req
$ /usr/lib/rpm/perl.req foo.pl perl(Foo::Bar) perl(parent)
$ perl -MDevel::Modlist=nocore foo.pl Can't locate Foo/Bar.pm in @INC (you may need to install the Foo::Bar module) (@INC contains: /usr/lib/bedrock/perl5 /opt/perl-5.28.1/lib/site_perl/5.28.1/x86_64-linux /opt/perl- 5.28.1/lib/site_perl/5.28.1 /opt/perl-5.28.1/lib/5.28.1/x86_64-linux /opt/perl -5.28.1/lib/5.28.1) at /opt/perl-5.28.1/lib/5.28.1/parent.pm line 16. BEGIN failed--compilation aborted at foo.pl line 4.Okay, so the errors are clearly indicative of compiling and then dumping @INC I suppose. So it appears we have at least 2 different methods of finding Perl dependencies.
- compile and dump @INC
- parse Perl and do semantic checking
$ echo "use LWP;" > foo.pl
$ scandeps.pl foo.pl | wc -l 96
$ scandeps.pl -R foo.pl | wc -l 1scandeps.pl, no recursing, with compiling
$ scandeps.pl -Rc foo.pl | wc -l 13
$ perl -MDevel::Modlist=nocore foo.pl | wc -l 13
$ /usr/lib/rpm/perl.req foo.pl | wc -l
So the direct dependency for foo.pl we know is only LWP and both /usr/lib/rpm/perl.req and scandeps.pl without compiling can figure that out, however scandeps.pl does get tripped up with our pathological case:
use parent qw/Foo::Bar/;Why does any of this matter anyway? I'm not sure honestly, however a the end of the day I do want to make sure I articulate module dependencies in my CPAN distribution correctly.
While it's true using the options that recurse and compile will produce a more complete listing of dependencies it seems redundant (and possibly problematic if we start talking about module versions and what's actually in your current Perl environment). If every CPAN module lists its direct dependencies then tools like cpanm should have no problem resolving all dependencies for a given Perl module distributed on CPAN. Correct?
So it seems it would be "best" to list only your direct dependencies and hope everyone else has a complete list of their direct dependencies.
To do that (list direct dependencies only), from the tools I've looked at so far (which include others I have not discussed in this blog), /usr/lib/rpm/perl.req did the best job of telling me what the direct dependencies of my module are.
My gut tells me I am missing something and that the greater Perl community as always has the answers as to what's the best practice for describing dependencies in a CPAN distribution.
P.S. This is the most awfulest blog platform every invented. It's almost impossible to determine how to format things as code or fixed fonts despite the various allegedly supported formats - not to mention the maddening need to reset my password every time I log in (to the same damn password!). It would be nice if Perl had a central place where the community could blog that actually worked well.