Deploying Perl code with git, local::lib, Minicpan and cpanminus
At $work we're nearing the end of a long upgrade process, that's included moving from Debian etch to lenny, from svk to git, from Perl 5.8 to 5.10, from Apache 1.3 to Apache 2 (and then to Plack), and moving to recent versions of DBIC and Catalyst (which had previously been pegged at a pre-Moose state).
For code deployment we had been packaging all Perl modules as Debian packages and keeping our own CPAN repository, but wanted a more 'Perl-ish' solution avoiding the various pitfalls with OS packaging and Perl.
THE STRUCTURE
The system that we implemented consists of the following parts:
- a main git repository for code development - with a 'stage' branch for code ready to be pushed out to a stage server (and then to live)
- a git repository for Minicpan - with two branches, dev and master
- a git repository for a development local::lib
- a git repository for a master ('live') local::lib
- a build server - with the main and Minicpan repositories checked out, and both dev and master branches of the local::lib repository checked out into separate directories.
(Click to enlarge)
THE PROCESS
The new build process works as follows:
- code is developed in a git branch, and merged into the final 'stage' branch when ready
- a 'bump' script is run to ready a distribution for deployment. This does the following:
- on the user's local machine:
- checks that everything is checked in and commited in git
- runs the tests (via prove)
- updates the package version number and commits and pushes the change to origin
- on the build server
- updates the git repository on the build server
- runs the tests and makefile on the build server
- 'injects' the distribution into a new branch in the minicpan repository - branch is named with package, version and date
- merges this branch into the dev branch
- uses cpanminus to install the distribution into the development local::lib repository
- on the user's local machine:
Once packages are installed into the local::lib they can be deployed onto any development server via rsync.
The process of deployment to a live server (our 'push live' script) works as follows:
- on the build server, retrieves the list of branches in the minicpan repository. Each branch is a new version of a package
- uses this list (with package, version, date) to prompt the user to select which distributions are to be pushed to live. The prompt also contains a link to Gitalist to show the relevant code changes in the update.
- merges any chosen branches into the master branch of the minicpan repository
- deletes chosen package branches (so they no longer appear in the selection list)
- uses cpanminus to install the distribution into the master local::lib repository
Then once packages are installed into the master local::lib they can be deployed onto any live server via rsync (We rsync the whole of the relevant lib, man and bin directories). We can deploy updates to individual machines, or server groups, but aim to update every server at least weekly.
The 'rollout' script can also run apt-get, restart webservers, run Puppet, etc, as required.
CPAN MODULES
The system allows modules from CPAN to be deployed by using a script to inject a distribution file (tar.gz) into a new branch in the Minicpan git repository. The branch is titled with the name, date and version, and can be selected during the 'push live' stage to be deployed to the live servers.
An update script uses the minicpan update_mirror() feature to download the latest distribution files, so that they can be injected if an update is required.
GIT
We looked at Git modules on CPAN, but ended up just writing our own library, calling the commands with IPC::Run3. Other than git's inconsitent usage of STDOUT and STDERR this has been a simple process.
NOTES
We use cpanminus with the 'mirror' and 'mirror-only' options, to specify that all dependencies should be pulled in from our local minicpan repository. Using cpanminus means that dependencies are automatically pulled in easily - the occasional module that causes problems for cpanminus (for example, one with a non-standard version numbering system) can be injected manually by finding the distribution file.
Note that our situation is simpler than other organisations:
- small development team
- all code runs on our own hardware (no third-party deployment)
- all hardware runs the same OS (and same version)
- only one version of a package needs to be live at any time (while different servers can be running different versions, this is never required for long periods, only during a bugfix for example).
- all development machines can have the same version of a package installed (which can be overridden locally)
I blog about Perl.
As the author of Git::Repository, I'd be interested in knowing the reasons why none of the existing Git wrappers (Git.pm, Git::Wrapper, Git::Class and Git::Repository -- the last 3 of them available on CPAN) didn't suit your needs.
This was a while ago now (last summer), and I'm not sure we looked at Git::Repository, but I do recall git's STDOUT/STDERR mess being a bit of an issue - depending on the command, we sometimes need to parse STDOUT and sometimes STDERR to get useful responses, similarly some STDERR outputs should result in a fatal error while others should be just ignored, so we would have ended up having to wrap commands made with one of the existing Git modules anyway. We don't have any strong feelings on this, it's potentially something that will change, at the moment it all works, so it's being left as-is.
We should have probably looked more at Git error codes too, I'm not sure finding documentation for them was easy at the time.
The first one is available on CPAN too. I took the Git.pm out of the Git sources and turned it into a CPAN module so I could use it as a dependency. It doesn't need anything else in the git sources.
This is excellent, excellent work! This exact kind of setup is something I know I've I've been trying to get more people to do. I know lots of people have set things up like this, using the combo of local::lib+git+minicpan, but there's still quite a bit of details to get it all working... Thank you for posting about what you did!
I did a talk on this kind of thing at YAPC::NA last year (see http://perl.scaffidi.net/home/cpanhell ) and I really hope I inspired some people to start using these techniques... and I hope people your blog and see just how it all fits together. It's just awesome to see these tools and practices being put to use by other people out there!
Thanks! yes, the system has been working very well for us.
Something that I should have mentioned is regarding updating MiniCPAN - I had to delete the CPAN Index files each time before the update, otherwise they were not picked up as being out of date (the file modification times, I think) since they'd just been checked out of Git.