After breaking the build twice in the space of a week for more or less the same reason each time, I figured that my team needed a continuous integration rig, and we needed it now.

For web application stuff it's not that unusual to have tests that collide with each other, at least in the short term due to the conflict between the need to "do it right' and the need to "do it now". And generally it's desirable to run your test suite in parallel wherever possible. At the moment, our tests are generally inadequate, so I'm only saving around 2 minutes on a parallel versus series run. However I'm doing bulk changes (with the help of PPI) to this largeish existing codebase at the moment, and get into situations where I want to run the whole test suite every 10 minutes or so, at which point saving 12 minutes in an hour becomes significant.

Now that we have a convenient testing rig, I expect the number of tests to increase exponentially, so sorting out parallel versus series early in the process means we have one less problem the next time some test infrastructure issue arises.

So I came up with this short rig based on App::ForkProve so that parallel and series test results are emitted in the same TAP output. To mark a test as only suitable for running in series, the text SERIES_TEST should be present in the file. Or you can pick some other distinguishing string.

#!/usr/bin/env perl;
use strictures 1;
use IPC::System::Simple qw/capture/;

use FindBin qw/$Bin/;
# use lib::require::all qw{modules lib $Bin/t/lib}; # would be nice but not working right now.
use App::ForkProve;

=head2 run_tests.pl

Runs all parallellisable tests 9 at a time, then runs the serial
tests separately, sending all to the same TAP report.

# TODO consider if adding an option for -v makes sense at all

=cut

use File::chdir;
$CWD = $Bin;

my @parallel = capture ("ack -L --perl --ignore-dir t/data --ignore-dir t/lib SERIES_TEST t");
my @series = capture ("ack -l --perl --ignore-dir t/lib --ignore-dir t/data SERIES_TEST t");

chomp $_ for @series;
chomp $_ for @parallel;

App::ForkProve->run(qw/-j 9/, @parallel);
App::ForkProve->run(qw/-j 1/, @series);

Came across this in my travels this morning:

$ perl -E 'sub x { say "y"}; my $x = 'x'; $x->();'
y
Undefined subroutine &main::1 called at -e line 1.

Why on earth does this compile and run in the first place? Fixing the shell quotes makes the problem go away.