Deploying Perl code with git, local::lib, Minicpan and cpanminus
At $work we're nearing the end of a long upgrade process, that's included moving from Debian etch to lenny, from svk to git, from Perl 5.8 to 5.10, from Apache 1.3 to Apache 2 (and then to Plack), and moving to recent versions of DBIC and Catalyst (which had previously been pegged at a pre-Moose state).
For code deployment we had been packaging all Perl modules as Debian packages and keeping our own CPAN repository, but wanted a more 'Perl-ish' solution avoiding the various pitfalls with OS packaging and Perl.
The system that we implemented consists of the following parts:
- a main git repository for code development - with a 'stage' branch for code ready to be pushed out to a stage server (and then to live)
- a git repository for Minicpan - with two branches, dev and master
- a git repository for a development local::lib
- a git repository for a master ('live') local::lib
- a build server - with the main and Minicpan repositories checked out, and both dev and master branches of the local::lib repository checked out into separate directories.
The new build process works as follows:
- code is developed in a git branch, and merged into the final 'stage' branch when ready
- a 'bump' script is run to ready a distribution for deployment. This does the following:
- on the user's local machine:
- checks that everything is checked in and commited in git
- runs the tests (via prove)
- updates the package version number and commits and pushes the change to origin
- on the build server
- updates the git repository on the build server
- runs the tests and makefile on the build server
- 'injects' the distribution into a new branch in the minicpan repository - branch is named with package, version and date
- merges this branch into the dev branch
- uses cpanminus to install the distribution into the development local::lib repository
- on the user's local machine:
Once packages are installed into the local::lib they can be deployed onto any development server via rsync.
The process of deployment to a live server (our 'push live' script) works as follows:
- on the build server, retrieves the list of branches in the minicpan repository. Each branch is a new version of a package
- uses this list (with package, version, date) to prompt the user to select which distributions are to be pushed to live. The prompt also contains a link to Gitalist to show the relevant code changes in the update.
- merges any chosen branches into the master branch of the minicpan repository
- deletes chosen package branches (so they no longer appear in the selection list)
- uses cpanminus to install the distribution into the master local::lib repository
Then once packages are installed into the master local::lib they can be deployed onto any live server via rsync (We rsync the whole of the relevant lib, man and bin directories). We can deploy updates to individual machines, or server groups, but aim to update every server at least weekly.
The 'rollout' script can also run apt-get, restart webservers, run Puppet, etc, as required.
The system allows modules from CPAN to be deployed by using a script to inject a distribution file (tar.gz) into a new branch in the Minicpan git repository. The branch is titled with the name, date and version, and can be selected during the 'push live' stage to be deployed to the live servers.
An update script uses the minicpan update_mirror() feature to download the latest distribution files, so that they can be injected if an update is required.
We looked at Git modules on CPAN, but ended up just writing our own library, calling the commands with IPC::Run3. Other than git's inconsitent usage of STDOUT and STDERR this has been a simple process.
We use cpanminus with the 'mirror' and 'mirror-only' options, to specify that all dependencies should be pulled in from our local minicpan repository. Using cpanminus means that dependencies are automatically pulled in easily - the occasional module that causes problems for cpanminus (for example, one with a non-standard version numbering system) can be injected manually by finding the distribution file.
Note that our situation is simpler than other organisations:
- small development team
- all code runs on our own hardware (no third-party deployment)
- all hardware runs the same OS (and same version)
- only one version of a package needs to be live at any time (while different servers can be running different versions, this is never required for long periods, only during a bugfix for example).
- all development machines can have the same version of a package installed (which can be overridden locally)