Of course I still use Perl

This post describes my current practice. Input/suggestions are most welcome to improve the process.

I am presently maintaining about 300 Perl distributions, 90% of which are CPAN distributions. As mentioned on a previous blog post, I am now developing some non-CPAN Perl software projects à la CPAN, by grouping software components as Perl distribution and releasing them like CPAN distributions, although not necessarily to CPAN. That number is expected to shoot up to at least 350-400 in 6-12 months, as more code is converted into Perl distributions. So without some helper scripts and automation, one can quickly go insane.

The basics: each dist is put in its own git repository (I used to put every Perl dist in a single big git repo, but this does not work well with git as the size becomes rather too big and it's harder to share/collaborate on github, etc). All dist is managed by Dist::Zilla (any dist builder tool is fine actually, dzil just happens to be one of the earliest and the most popular tool). I name all repos perl-Foo to differentiate it from other non-Perl-dist repos (I happen to put all my git repos under a single directory and manage them with Git::Bunch). Other convention like p5-Foo or Foo-pm is equally fine, as long as you're consistent.

I have a script to list all my git repos and my Perl dists and filter them according to some criteria. This helps me if I want to select all/some repos to push to github, or do whatever. The criteria include: which project the dist belongs to (either CPAN, or other project). Since CPAN projects are still the majority, all Perl dists are assumed to be a CPAN dist unless a tag file called .tag-proj-PROJ on the top-level directory exists. All CPAN projects are assumed to be public repos. A dist which have a project tag file is assumed to be non-public unless a tag file .tag-public also exists. Public repos are those that are pushable to github/bitbucket/gitorious. Non-public repos are those that should only be sync-ed to personal backups or company server.

Another criteria is whether I have done releases on a certain dist. Dists that have never been released are marked with tag file .tag-unreleased. They get more relaxed treatment (e.g. some I have not written dist.ini or weaver.ini for).

I might also add other tag files to categorize the dists further.

The script can also filter repos that have modified or untracked files, or that are recently modified in the last X days (by looking at the most recent 'git log' entry).

For each project, I have a single dzil plugin bundle and pod weaver plugin bundle, to make things uniform across a project. For CPAN project, this is Dist::Zilla::PluginBundle::Author::SHARYANTO and Pod::Weaver::PluginBundle::Author::SHARYANTO. For project X, for example, I also have a corresponding Dist::Zilla::PluginBundle::Project::X and Pod::Weaver::PluginBundle::Project::X.

My dzil plugin bundle is pretty simple/conservative. First of all, I avoid stuffs that change line numbers because this makes debugging much more cumbersome. So for example I choose OurPkgVersion instead of the default PkgVersion plugin. I also put POD, #ABSTRACT, #PODNAME at the end. Some other things that I do: check Changes, add a compile test.

My Pod::Weaver plugin bundle is also pretty simple. Aside from the default boilerplate generated by default Pod::Weaver, I add HOMEPAGE, SOURCE (repository), and BUGS sections. For CPAN projects, they point to MetaCPAN, github, and RT. My other non-CPAN public repos also point to github by default, but to project-specific bug tracker and homepage.

For non-CPAN projects, I remove the ConfirmRelease and UploadToCPAN plugins.

I have another script to release dist with a single command. Actually this command can just be dzil release because dzil has a plugin to run stuffs during different phases, but I grew this script before I use the Run::* plugins so this script calls 'dzil release' and do extra stuffs outside of it too. The most notable extra thing this script does is to archive the release tarball to my release archive repos. I archive my Perl release tarballs along with a record using LTSV format. An example for my CPAN projects: releases-perl-cpan and releases.txt.

I have yet another script to list my Perl releases and filter them according to some criteria. This lets me find out how many releases I've made over the last {week,month,year}. The criteria include project name, new dist, trial/dev, version number/range, date.

That's it so far. Things I haven't explored include:

• setting up continuous integration;
  
• setting up a CPANTesters-like infrastructure;
  
• automation in preparing software releases that bundle the different and relevant Perl distributions as a single final release.