Perl::QA Hackathon in Lyon - Day 1
I'm at the Perl QA Hackathon in Lyon and it's been an interesting trip so far. I missed my flight yesterday, so I had to fly out this morning — only to get to the airport and discover that I left my passport at home. Fortunately, the hackathon is in Lyon, France, so I was able to use my Titre de Sejour (residence permit) instead. Then the coffee machine ate my money.
Then shortly after I get to the hackathon (after four hours of sleep followed by four hours of travel), Leon Timmermans hit me with an interesting problem regarding parallel tests in
Test::Harness. I came up with an approach that isn't as sexy as his, but is far simpler and it involves a module I released today, TAP::Stream.
When we first discussed nested TAP a few years ago, one use case was the ability to merge multiple TAP streams into a single stream via subtests. In other words, these two streams could be combined:
# stream 1 1..2 ok 1 - some test ok 2 - another test # stream 2 ok 1 - foo ok 2 - bar not ok 3 - baz 1..3
And the merged streams should look like this:
1..2 ok 1 - some test ok 2 - another test ok 1 - stream 1 ok 1 - foo ok 2 - bar not ok 3 - baz 1..3 not ok 2 - stream 2 1..2
Why would you want to do that?
Maybe you've distributed your tests across multiple machines and want to combine the TAP output. Maybe you're running several processes and want to combine their TAP output (a variant of the first reason). Maybe you've saved some TAP somewhere and want to read it in and combine with more TAP you have somewhere else (or are currently generating).
It's an obscure use case, but if you have this need, it's easy to get this wrong.
The interface looks like this:
use TAP::Stream; use TAP::Stream::Text; my $tap1 = <<'END'; ok 1 - foo 1 ok 2 - foo 2 1..2 END # note that we have a failing test my $tap2 = <<'END'; ok 1 - bar 1 ok 2 - bar 2 1..3 ok 1 - bar subtest 1 ok 2 - bar subtest 2 not ok 3 - bar subtest 3 #TODO ignore ok 3 - bar subtest not ok 4 - bar 4 1..4 END my $stream = TAP::Stream->new; $stream->add_to_stream( TAP::Stream::Text->new( name => 'foo tests', text => $tap1 ), TAP::Stream::Text->new( name => 'bar tests', text => $tap2 ) ); print $stream->to_string;
And that outputs:
ok 1 - foo 1 ok 2 - foo 2 1..2 ok 1 - foo tests ok 1 - bar 1 ok 2 - bar 2 1..3 ok 1 - bar subtest 1 ok 2 - bar subtest 2 not ok 3 - bar subtest 3 #TODO ignore ok 3 - bar subtest not ok 4 - bar 4 1..4 not ok 2 - bar tests # Failed 1 out of 4 tests 1..2
Of course, instead of just adding text to a stream, you can also add streams to streams and it Does The Write (sic) Thing. I use this inside of Test::Class::Moose to allow for parallel testing.
Test::Harness, it won't work directly because I use Moose internally to add some type safety (a fine decision since it was for a Moose-based testing framework), but it should be easy to hack to only use core modules. However, the problem it solved involved two very fascinatingly different ways of approaching a problem.
If you run
Test::Harness in parallel, you have a single process spawning multiple other processes and reading their output. Unfortunately, the issue there is that if the read blocks on one process, you wait and don't read the others. You could potentially wait a long time to get your results if this happens, and then they potentially happen at once. Oops. Leon's solution was to invert how things work. Instead of the parser using a grammar which reads results from processes, you'd have processes sending events to the grammar which would, in turn, send results to the parser. If one process blocks, the others could still send events.
That's great, but it could involve rewriting a bunch of code.
Test::Class::Moose, the parent process doesn't read from the children. Instead, the children write to the parent's
TAP::Stream object, making the code much simpler and sidesteps the problem of blocking on a child's write.
TAP::Stream is also on github, of course.
Also, Ricardo Signes cleaned up my
dist.ini and explained why my cargo-culted crap was, well, crap. Thanks Ric!