We all know that Perl unit tests, aided by Test::More and prove produce results in TAP format. But since Jenkins is Java and someone once dictated that Java shall only read XML and only write XML, Jenkins expects test results to conform to the JUnit XML format. Well, OK, if they want XML, let's give them XML. So you turn to your favorite search engine and ask it how to transform TAP to JUnit. You will, of course, find something. And since that something was actually written by Justin Mason of SpamAssassin fame, you stop searching at that point and download that conversion script.

Well, the first mistake was to use Google. It's always better to search CPAN first in a case like that. A quick look at those CPAN search results goes a long way. It turns out that you don't have to transform TAP to JUnit, you can simply tell prove to use a different formatter for its output:

prove --formatter=TAP::Formatter::JUnit -l t > test_results.xml

It's really that simple. Now go ahead and look at the documentation for prove. It's full of goodness! For example, you might want to add --timer to the line above, and you will not only see which tests failed or didn't, but also how long each test took.

prove --timer --formatter=TAP::Formatter::JUnit -l t > test_results.xml

There is really no need to save temporary TAP output and then transform it into JUnit later.

Moving along, you will find that some people out there are actually trying to tell you that you should run a find to gather your test files and then loop over the results to pass them to prove. Of course that's possible, but it's also quite clumsy, completely unnecessary and it will run your tests in no particular order. So don't do this:

for t in $(find t -name '*.t') ; do prove ....; done

prove will find you test files if you tell it to, simply use -r for that.

prove -r --timer --formatter=TAP::Formatter::JUnit -l t > test_results.xml

Still relying on Google, you might come up with some clever Unix sexyness:

prove -r --timer --formatter=TAP::Formatter::JUnit -l t | tee test_results.xml

I guess people use the tee because that lets them see the test output when looking at the console output of their build jobs, but tee has a nasty side effect: It will hide prove's exit value from Jenkins. Your builds will not fail when you tee their output. At worst, they will be marked as unstable. So if you want your build to fail when some of your tests fail, stay away from tee! Use it for the initial configuration of your jobs to make sure you don't miss any error messages in the console output, but then get rid of it.

Perl is elegant. Or can be. For me, what I learned about prove when working with Jenkins, confirmed this once again.

This is what I started with:

for t in $(find t -name '*.t') ; do prove -l $t 2>&1 | tee -a $OUTPUT/test-$BUILD_NUMBER.tap; done
tap-to-junit-xml --input=$OUTPUT/test-$BUILD_NUMBER.tap --output=test_results.xml

And this is what I ended up with:

prove -r --timer --formatter=TAP::Formatter::JUnit -l t > test_results.xml

Not only is this easier to read, it also does what I want.

But why is code less readable when it's full of parenthesis, e.g.? I think the answer is simply that the puctuation lowers the signal-to-noise ratio. When reading code, you are looking for the signals, the statements that actually do something, the meat, the beef. A corollary of this is that a bit of whitespace here and there will increase your code's signal-to-noise ratio because it will make the beef stand out.

Here's a stupid little example:

$x=foo();
if(defined($x)) {
    bar($y,$x,$z);
}

That's pretty cramped code and whitespace will make it more readable:

$x = $foo();
if ( defined( $x ) ) {
    bar( $y, $x, $z );
}

It's a lot easier to see what is is that needs to be defined here and what the arguments passed to bar() actually are.

Now, assuming this is possible here, let's get rid of the excessive punctuation:

$x = foo;
if ( defined $x ) {
    bar $y, $x, $z;
}

Even better! Readable! You don't have to concentrate too hard to see what is going on here.

All this got me thinking, that at this low level of symbols, code readability should be measurable. If punctuation increases the noise and whitespace increases the signal, it should be easy to come up with a formula here. Let's try that, with those examples:

$x=foo();

Tokens you need to understand: 3 ($x, =, and foo). Amount of whitespace: 0. Punctuation: 2 (the parens).

So that might be a SNR of ( 3 + 0 ) / 2 = 1.5.

$x = $foo();

Tokens: 3, whitespace: 2, puctuation: 2. This gives us ( 3 + 2 ) / 2 = 2.5.

And finally:

$x = foo;

which results in a divide by zero exception! In this rare case, this is exactly what we want.

I wonder whether something like this could be turned into a Perl::Critic policy? Or does is already exist?

PS: What I really don't get about "Effective Perl Programming" is the unexplained insistence to put the lexical file handle variable with three-argument open in parentheses. As in open my ($foo), '<', 'foo.bar'.