Essentially you set up a git post-commit hook that causes your tests to get run on every commit, against a number of different Perl versions, with databases and other services available if needed. And it's all free!
If you visit https://travis-ci.org/ you can get a feel for the interface and the tests that are being run. For a particular commit you get a build, for example WebService::Nestoria::Search build 1, which has a sub-build per Perl version, for example WebService::Nestoria::Search build 1.1 (perl 5.16). As you can see you get the full output from the Ubuntu VM that's running your tests, so if anything does go wrong it's pretty simple to debug.
For the rest of this post I'm going to describe the integration process, in particular hitting on how to make it work with Dist::Zilla-based projects.
]]> Travis CI BasicsIf you're using Github this couldn't be easier. Log into Travis CI with your Github account, go to your Travis CI profile, and flip your projects from Off to On. That causes a Github-side post-commit hook to be set up so that every time you 'git push' up to Github a Travis build will run.
Then you need to set up your .travis.yml file and commit it to your repository. Here's an example from a "normal" (non-Dist::Zilla) distribution:
language: perl
perl:
- "5.16"
- "5.14"
- "5.12"
- "5.10"
That's it, that's the whole file! Travis knows from 'language: perl' that it should look for a Makefile.PL or Build.PL, install the dependencies, and then run the tests.
Once you commit that you should be able to head back to Travis CI and see the "My Repositories" tab and see that your tests are either running or have run.
Travis CI + Dist::Zilla
At the time of writing Travis CI does not support Perl distributions that don't have either a Makefile.PL or a Build.PL file. However there are a few different options for how to support a project that only has a dist.ini file...
Add this functionality to Travis CI
Travis is open source, you can see (and fork) it here: https://github.com/travis-ci
I tried adding Dist::Zilla support and failed, mostly due to my lack of Ruby :-) I suspect if somebody headed on IRC and talked to rjbs and miyagawa a solution could be found, since they both got involved on Github when I made my attempt and were very helpful in pointing out my errors and showing me towards the ways below.
Tell Travis CI to do the "right" thing for your project only
The .travis.yml file is incredibly flexible, up to and including allowing you to run arbitrary commands before and instead of the test process. That means if you make your .travis.yml file look like this...
language: perl
perl:
- "5.16"
- "5.14"
- "5.12"
- "5.10"
before_install:
# Prevent "Please tell me who you are" errors for certain DZIL configs
- git config --global user.name "TravisCI"
install:
# Deal with all of the DZIL dependancies, quickly and quietly
- cpanm --quiet --notest --skip-satisfied Dist::Zilla
- dzil authordeps | grep -vP '[^\w:]' | xargs -n 5 -P 10 cpanm --quiet --notest --skip-satisfied
- export RELEASE_TESTING=1 AUTOMATED_TESTING=1 AUTHOR_TESTING=1 HARNESS_OPTIONS=j10:c HARNESS_TIMER=1
- dzil listdeps | grep -vP '[^\w:]' | cpanm --verbose
script:
- dzil smoke --release --author
... then your dist.ini-based module will be tested correctly, by causing it to run the appropriate cpanm and dzil commands to both install dependencies and run the tests themselves.
The Dist::Zilla plugin TravisYML (Dist::Zilla::TravisCI) will generate the file above automatically as part of your 'dzil build' if you want to do it that way.
The downside is that installing Dist::Zilla every time is very slow so your tests will take a long time. Luckily there's another way...
3. Use Git::CommitBuild and Travis CI branch whitelisting
The Dist::Zilla plugin Git::CommitBuild (Dist::Zilla::Plugin::Git::CommitBuild causes your build to be committed on a separate branch each time you run 'dzil build', or a command which implies 'build' such as 'dzil test' or 'dzil release'.
Add the plugin to your dist.ini like so:
[Git::CommitBuild]
release_branch = build/%b
release_message = Release build of v%v (on %b)
With that you get this:
alex@yuzu:~/Documents/Git/App-highlight$ git branch
build/master
* master
alex@yuzu:~/Documents/Git/App-highlight$ ls
bin Changes dist.ini lib README.pod t
alex@yuzu:~/Documents/Git/App-highlight$ git checkout build/master
Switched to branch 'build/master'
alex@yuzu:~/Documents/Git/App-highlight$ ls
bin Changes dist.ini lib LICENSE Makefile.PL MANIFEST META.json META.yml README README.pod t
As you can see the 'build/master' branch does have a Makefile.PL file, so Travis CI will be able to run its normal Perl testing against it. All that's left is to update your .travis.yml to tell Travis to run the tests on that branch only, like so:
language: perl
perl:
- "5.16"
- "5.14"
- "5.12"
- "5.10"
branches:
only:
- /^build/
You also need your .travis.yml file to exist on the build/master branch. To do that you need to update your dist.ini's GatherFiles and PruneCruft plugin options like so:
[GatherDir]
include_dotfiles = 1
[PruneCruft]
except = \.travis.yml
And with your next commit (and push to github if that's where your Travis CI hook is set up) a Travis build should be kicked off!
Useful Links
Travis CI:
Examples:
Top line summary: it was brilliant!
This was the second course I've attended from Dave Cross, and the first one under the Perl School banner. Dave has been using Perl heavily for around two decades up to and including this year, which gives him a very deep knowledge that is also firmly up-to-date. He is also an excellent presenter and trainer, so all that knowledge I just mentioned comes pouring out freely in a way that is easy to understand.
I would say that Perl School is aimed at people more towards the "beginner" end of the spectrum, but the courses are focussed enough that as a Perl expert but a DBIC beginner I found it informative and interesting. I suspect that others would find the same holds for the "Object Oriented Programming with Perl and Moose" course that is also available. This is not a bad thing - I took this opportunity to bring along one of the less senior members of my team at Lokku, and I recommend others do the same. It also means that, unlike YAPC or LPW, the room was full of faces I didn't recognise. If those people can be converted into YAPC attendees then our community is going to grow and thrive.
I also know that Dave is looking to add more courses and to constantly improve Perl School, as is evident from the post-course survey and the conversations during the day itself. If you have any requests, comments, or ideas for courses please do contact him over on the Feedback form on the site.
If you're thinking to yourself that a course on Moose or DBIx::Class sounds great then you're in luck - Perl Schools 5 and 6 will be repeat performances of the Moose and DBIC courses, on April 6th and June 8th. At only £30 they are a bargain, there's really no excuse for not going!
I for one am really looking forward to seeing what happens with Perl School over the next year or so. I'm expecting great things :-)
]]>Why?
Not sure which to be more scared about - getting zero questions or getting lots of questions! But I'm more excited about getting lots of questions.
Whether you're completely new to Perl, or a Perl expert with a random question, you can always Ask JAPH!
Let's see how this goes :-)
]]>Hi guys and girls,
We're looking for a Perl developer with 2+ years of experience programming
professionally to join our engineering team in central London, primarily to
work on the Nestoria property search engine (http://www.nestoria.com)
Nestoria is a great product to work on. As a vertical search engine we work
hard to solve many of the same problems as a larger search company:
We are looking for somebody who has:
We're offering c. £45K plus bonus.
If that sounds interesting to you check out the full job ad here:
http://www.lokku.com/jobs/experienced-perl-developer.html
If you have any questions or if you'd like to apply please get in
touch at perldeveloper@lokku.com
I'll also be at the London.pm technical meet up this coming Tuesday if
you'd like to catch me in person :-)
Thanks,
- Alex
]]>Cross-posting this from a Stack Overflow Question asked by a colleague.
Some processes that run for multiple hours (ETL jobs) suddenly started consuming a lot more RAM than usual. Analysis of the changes in the relevant release is a slow and frustrating process. I am hoping to identify the culprit using more automated analysis.Our live environment is perl 5.14 on Debian squeeze.
I have access to lots of OS X 10.5 machines, though. Dtrace and perl seem to play together nicely on this platform. Seems that using dtrace on linux requires a boot more work. I am hoping that memory allocation patterns will be similar between our live system and a dev OS X system - or at least similar enough to help me find the origin of this new memory use.
This slide deck:
https://dgl.cx/2011/01/dtrace-and-perl
shows how to use dtrace do show number of calls to malloc by perl sub. I am interested in tracking the total amount of memory that perl allocates while executing each sub over the lifetime of a process.
Any ideas on how this can be done?
We make great use of NYTProf for profiling the speed of our processes, but unfortunately there doesn't seem to be anything anywhere near that good for memory consumption.
Does anybody have any advice for "profiling" memory use in Perl?
Thanks!
]]>I'm at work right now, waiting for some code to run in the Perl Debugger and I got to thinking. Right now I've set a break point and I've hit "c" for continue executing, and in the background the debugger is doing all these checks on every single line and slowing down the execution of my program like crazy. I know it's doing something because if I hit ^C is halts properly at its current point of execution. What if there were some way to speed this up?
I'm thinking...
DB > b $condition
DB > $DB::breakpoints_only_mode = 1;
DB > cSetting $DB::breakpoints_only_mode essentially promises the debugger that you won't hit ^C or anything silly like that. The implementation would be something like...
What do Smarter People Than Me think of this idea?
(of course it's entirely possible that the debugger isn't actually slowing down my code all that much... but it feels like it is?)
]]>I just got back from FOSDEM 2011 last night and had an absolute blast, especially on Sunday when I spent the whole day in the Perl devroom. Thanks to everyone who came!
I gave a talk called Introduction to writing readable and maintainable Perl and I thought I'd share my slides here.
http://www.slideshare.net/kaokun/introduction-to-writing-readable-and-maintainable-perl
Unfortunately Google Docs => PDF => Slideshare isn't an entirely flawless reproduction, but most of the slides look ok.
I'm considering giving an updated version of the talk at YAPC::EU this year. If you have any feedback please leave a comment or send me an email :)
- Alex
]]>highlight.pl is designed to be piped into, like grep or ack, but instead of filtering out lines which don't match it just highlights words which do, similar to "ack --passthru" except that you get different coloured highlights for your different matches.
#!/usr/bin/perl
#
# pipe text to highlight.pl to highlight certain words
#
# eg. cat nastyfile.pl | highlight words you want
#
use strict;
use warnings;
use Term::ANSIColor qw/:constants/;
use Getopt::Long;
my ($no_escape,$full_line);
GetOptions(
'no-escape|n' => \$no_escape,
'full-line|l' => \$full_line,
'help|h' => sub { print usage(); exit 0 },
);
my @matches = (".+");
if (scalar @ARGV) {
if ($no_escape) {
@matches = @ARGV;
}
elsif ($full_line) {
@matches = map { qr/^.*\Q$_\E.*$/ } @ARGV;
}
else {
@matches = map { "\Q$_" } @ARGV;
}
}
my @colors = map { BOLD $_ } (
RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN
);
my $reset = RESET;
while () {
my $i = 0;
foreach my $m (@matches) {
s/($m)/$colors[$i] . $1 . $reset/ge;
$i++; $i %= @colors;
}
print;
}
exit 0;
##########################################
sub usage {
return <<" END";
$0
--no-escape [n] don't autoescape input, for using regular expressions
--full-line [l] highlight the entire line that contains the match
--help [h] show this message
END
}
As a bonus, here's an alias I find very useful:
alias svndiff="svn diff | highlight -n '^[-][^-].*$' '^[+][^+].*$' '^(([-+]{2,})|([=]{5,})).+'"
This highlights filename headers in yellow, added lines in green and deleted lines in red.
Feedback welcome :) This is the first time I'm showing this code so it's probably very rusty and inefficient (I just noticed that svndiff would be simpler to write if --full-line and --no-escape worked together!)
- Alex
]]>