Manage multiple MiniCPANs, and version them

Most of my work with DPAN revolves around the creation of private, CPAN-like repositories that a project team can use without affecting anyone else. Setting up a DPAN process for a recent customer involved making several MiniCPANs, one for each project group. I had to add a couple of features to CPAN::Mini to make it work out. These new features show up in CPAN::Mini 1.100590.

Like most CPAN tools, CPAN::Mini assumed that there would only ever be one repository, so a person tasked with maintaining several had a problem. Consider this workflow to support several groups:

  • There's a master MiniCPAN that holds all of the modules anyone in the company is allowed to use. The remote is a real CPAN.
  • Slave MiniCPANs pull from and filter the modules in the master to contain just the modules their application needs. The remote is the master MiniCPAN.
  • DPAN (or CPAN::Mini::Inject) adds project specific modules to the slaves.

I can subclass CPAN::Mini by specifying an implementing class, but that's the same problem I had with Making Module::Starrter Easier to Subclass: I don't get to insert my subclass until after the configuration processing has completed.

A single sysadmin maintaining all of this can now use the -C switch to minicpan to specify configuration files:

 % minicpan -C master.conf
 % minicpan -C projectA.conf
 % minicpan -C projectB.conf -c Local::CPAN::Mini::Filter::ProjectB

All of that can even be in a shell script so that everything updates in sequence.

That's all very nice, and is going to be very useful when new configuration options show up.

Along with that, I wanted to keep each of these MiniCPANs under source control. A developer could rollback to any version of their MiniCPAN so they could bisect it to find which module upgrades broke a particular feature. They could also checkout a particular version of their MiniCPAN but a source control tag. This also gives developers the chance to branch their MiniCPAN for experimental work while not forcing it on their workmates (although they can also make a slave MiniCPAN of their slave MiniCPAN), profile their applications against different sets of module versions, and so on.

That didn't work out so well previously because minicpan likes to remove files and directories, such as .svn, that aren't in the real CPAN. I've adjusted that too with a new configuration option:

 remote: http://www.example.com/CPAN/
 local: /CPAN
 ignore_source_control: 1

This is different than the existing option to skip_cleanup, which removes the old versions and so on. You can still have the cleanup, but the cleanup ignores files that look like they belong to source control. I have stuff in there for CVS, SVN, and Git, so if you need more, just make the patch. You can get the CPAN::Mini sources from Github but you can also edit files directly in Github so you don't have to do a lot of work for very minor changes like adding a filename to an array.

Now I need to go back and look at CPAN::Mini::Webserver to make it work with multiple MiniCPANs too. That is, it should be able to start off multiple instances as long as it knows which configuration file to use (which should be able to specify the port).

2 Comments

Seems that we had the urge to scratch the same itch: I've recently mashed CPAN::Cache and MyCPAN::App::DPAN with a Catalyst app to do roughly the same thing (http://babyl.dyndns.org/techblog/2010/03/). I'll have to look at your solution and see how much I can steal^D^D^D^D^D get inspired by it. :-)

Leave a comment

About brian d foy

user-pic I'm the author of Mastering Perl, and the co-author of Learning Perl (6th Edition), Intermediate Perl, Programming Perl (4th Edition) and Effective Perl Programming (2nd Edition).