Ovid

I'm at the Perl QA Hackathon in Lyon and it's been an interesting trip so far. I missed my flight yesterday, so I had to fly out this morning — only to get to the airport and discover that I left my passport at home. Fortunately, the hackathon is in Lyon, France, so I was able to use my Titre de Sejour (residence permit) instead. Then the coffee machine ate my money.

Then shortly after I get to the hackathon (after four hours of sleep followed by four hours of travel), Leon Timmermans hit me with an interesting problem regarding parallel tests in Test::Harness. I came up with an approach that isn't as sexy as his, but is far simpler and it involves a module I released today, TAP::Stream.

When we first discussed nested TAP a few years ago, one use case was the ability to merge multiple TAP streams into a single stream via subtests. In other words, these two streams could be combined:

# stream 1
1..2
ok 1 - some test
ok 2 - another test

# stream 2
ok 1 - foo
ok 2 - bar
not ok 3 - baz
1..3

And the merged streams should look like this:

    1..2
    ok 1 - some test
    ok 2 - another test
ok 1 - stream 1
    ok 1 - foo
    ok 2 - bar
    not ok 3 - baz
    1..3
not ok 2 - stream 2
1..2

Why would you want to do that?

Maybe you've distributed your tests across multiple machines and want to combine the TAP output. Maybe you're running several processes and want to combine their TAP output (a variant of the first reason). Maybe you've saved some TAP somewhere and want to read it in and combine with more TAP you have somewhere else (or are currently generating).

It's an obscure use case, but if you have this need, it's easy to get this wrong.

The interface looks like this:

use TAP::Stream;
use TAP::Stream::Text;

my $tap1 = <<'END';
ok 1 - foo 1
ok 2 - foo 2
1..2
END

# note that we have a failing test
my $tap2 = <<'END';
ok 1 - bar 1
ok 2 - bar 2
    1..3
    ok 1 - bar subtest 1
    ok 2 - bar subtest 2
    not ok 3 - bar subtest 3 #TODO ignore
ok 3 - bar subtest
not ok 4 - bar 4
1..4
END

my $stream = TAP::Stream->new;

$stream->add_to_stream(
    TAP::Stream::Text->new( name => 'foo tests', text => $tap1 ),
    TAP::Stream::Text->new( name => 'bar tests', text => $tap2 )
);

print $stream->to_string;

And that outputs:

    ok 1 - foo 1
    ok 2 - foo 2
    1..2
ok 1 - foo tests
    ok 1 - bar 1
    ok 2 - bar 2
        1..3
        ok 1 - bar subtest 1
        ok 2 - bar subtest 2
        not ok 3 - bar subtest 3 #TODO ignore
    ok 3 - bar subtest
    not ok 4 - bar 4
    1..4
not ok 2 - bar tests
# Failed 1 out of 4 tests
1..2

Of course, instead of just adding text to a stream, you can also add streams to streams and it Does The Write (sic) Thing. I use this inside of Test::Class::Moose to allow for parallel testing.

For Test::Harness, it won't work directly because I use Moose internally to add some type safety (a fine decision since it was for a Moose-based testing framework), but it should be easy to hack to only use core modules. However, the problem it solved involved two very fascinatingly different ways of approaching a problem.

If you run Test::Harness in parallel, you have a single process spawning multiple other processes and reading their output. Unfortunately, the issue there is that if the read blocks on one process, you wait and don't read the others. You could potentially wait a long time to get your results if this happens, and then they potentially happen at once. Oops. Leon's solution was to invert how things work. Instead of the parser using a grammar which reads results from processes, you'd have processes sending events to the grammar which would, in turn, send results to the parser. If one process blocks, the others could still send events.

That's great, but it could involve rewriting a bunch of code.

In Test::Class::Moose, the parent process doesn't read from the children. Instead, the children write to the parent's TAP::Stream object, making the code much simpler and sidesteps the problem of blocking on a child's write.

TAP::Stream is also on github, of course.

Also, Ricardo Signes cleaned up my dist.ini and explained why my cargo-culted crap was, well, crap. Thanks Ric!

If you're interested in hiring me or any of the talented developers we have working at All Around The World, drop me a line at ovid@allaroundtheworld.fr.