Building a cpanm-puppet-provider requires first a fuller-featured package manager for perl

My earlier post --
Seeking collaborators: native puppet cpanm provider has already attracted feedback and interest from four individuals on at least two continents.

Thinking out loud about creating a real puppet package provider to handle perl modules, led this afternoon to a conversation on #perl-help with ilmari_ who pointed out that the lack of a real package manager for perl will frustrate efforts to wrap that missing functionality in a native puppet provider for its package resources. He called what we have now 'module installation tools' which fail to rise to the level of a real package manager. At a minimum, he suggested a real perl package manager would also require the ability to uninstall a module, as well as the ability to query the system about the modules installed there, perhaps the ability to query upstream about available packages.

This immediately created a threat for scope creep as I now explore what would be required to provide that missing functionality, as prelude to writing a puppet provider which might wrap it.

But that might not be an insurmountable hurdle. alnewkirk on the irc channel pointed me toward: App::pmuninstall, which I installed this evening and will be testing next week (after a weekend trip).

This evening I outlined the features I imagine a perl module query tool might provide and invite your feedback on the subject. For the moment that list is posted on the github page for this project and below.

plenv list-modules already provides a list of modules installed in the current perl installation. cpan already can query its upstream repos for available modules. So can pinto. To that functionality I would want to add a means for querying with a cli tool, for:

module version # $VERSION of installed module
module version-latest # latest version available upstream
module installation path # absolute path to installation
module packlist-path # absolute path to .packlist
module deps # dependencies on perl modules
module deps-external # dependencies on non perl resources
module deps-testing # dependencies used for testing
module deps-build # dependenccies used for building module
module provides-commands # what cli tools are provided
module provides-modules # what modules are provided
module provides-methods # what public methods are exposed
module provides-functions # what public functions are exposed
module author # principal author(s)
module maintainer # and their contact information
module release-date # for the current version
module cpan-url # what it says
module metacpan-url # self-explanatory
module test-results # url to published smoke test results
module repo # url for a publically accessible vcs repository
module issues # open bug count and
# preferred means for issue tracking
module irc # server and channels supporting
# module users or development

So it seems that this project might actually be two projects:

(1) tying together the existing tools with some new code to provide a full-featured package manager for perl modules (pmpm, anyone? a perl module package manager, perhaps); and

(2) writing a puppet provider so that rather than faking this with Exec['install-Perl::Module'] resources, puppet administrators can use an honest and native Package['Perl::Module'] resource instead, honoring the paradigm of declaring state, so we can ensure => purged | present | latest and let puppet handle the details under the hood by calling the pmpm collection of tools to do the perl things in a platform appropriate way.

I'm counting on those who said they would use such a tool to help us test it when there is something worth testing. My appreciation for the feedback so far. I think this conversation could actually lead somewhere. I work in a linux environment and will depend on others to contribute to helping us build that cross-platform functionality.

Collaborators include not just those who send me pull requests for code, but also those who engage in this conversation to define a useful interface and to help me think through the problem space. Its been a stimulating, engaging and productive conversation so far, and it only started a few hours ago. I'm curious to know where it will go from here, to know what you have to add to the conversation.

5 Comments

I'd prefer simple and reliable tools to automatically build packages for Debian, Arch Linux and other popular operating systems (how about AppStore and Windows Installer?). Puppet can install Debian packages, so what's the point of this project?

It's not that easy to find all installed modules. We had a lengthy discussion about that in a German Perl forum...

* Not every installation path writes .packlist (installation via system package manager e.g. yum, apt-get, ..., manual "installation" by copying the .pm to a path in @INC)
* Do you want to list modules installed in a project specific path (local::lib)?
* ...

A notable difference between package managers and CPAN installers (from my limited knowledge/experience of both), is that packages are built for a specific platform (specific OS, arch, etc) so the package manager role is mostly to deal with file copy and deletion. (Once you've reached the install step. Dependencies and incompatibilities are actually more related to interacting with the package database.) A module installer, on the other hand, actually runs code (from the distribution to be installed) and generates the files that are going to be copied (e.g. compiles XS, but also produces the proper shebang line, etc.).

In other words, given a specific package file, its installation will copy the same files on your system than on any other system. Given a specific CPAN distribution, well, there's no real way to know before you actually run it.

If .packlist is unreliable, I guess it's time to design a SQLite db for local installs. Or is there one available already?

I think BooK said it best:

[The] package manager role is mostly to deal with file copy and deletion. A module installer, on the other hand, actually runs code (from the distribution to be installed) and generates the files that are going to be copied

I haven't done this myself, but I've heard that some shops use cpanm & pinto to build their Perl stack into a single wad (e.g. directory), and then wrap that with a package that depends on non-Perl resources (like databases and webservers).

That way, you can still use the OS package manager for deployment and you only have to make one package for your Perl stuff, instead of one for every modules.

Leave a comment

About hesco

user-pic I blog about Perl.