Ovid

I like writing code. I like writing tests. I don't like:

• Trying to figure out where the tests are
• Writing boilerplate
• Finding yet another package without tests

That last one is particularly vexing when you discover that your code is failing because another package doesn't load, but it doesn't have tests. So I fixed that.

Some of what follows is a repeat of things written in previous posts, but it's important enough that it bears repeating.

A couple of months ago I suggested we create "discoverable" tests at work. In short, many of us face the problem of trying to figure out where the tests are in a new project. Worst case is this:

$ ls t/*t | wc -l
312

And you're sitting there weeping because the nice, orderly code in your lib/ directory is tested by a pile 'o junk. Other projects have subdirectories, but those subdirectories often group things by behavior, not package; your tests for lib/Foo/Bar/Baz.pm wind up in t/common/exceptions/, t/common/standards/, t/database/, t/miscellaneous/crap/tests/. Even if you memorize all of those subdirectories, you find yourself adding code that can, arguably, be in multiple subdirectories.

My recommendation was that our modules have one corresponding .t file. The agreement reached was that a module like lib/Foo/Bar.pm would go into t/Foo/Bar.t. I also have the following in my .vimrc:

noremap ,gg :call GotoCorresponding(expand("%"))<cr>                                                                                     

function! GotoCorresponding(module)                                                                                                       
    let file = system("get_corresponding ".a:module)
    if !empty(file)
        let ignore = system("perl /home/cpoe/bin/make_test_stub ".a:module." ".file)
        execute "edit " . file
    else
        echoerr("Cannot find corresponding file for: ".a:module)
    endif
endfunction

The get_corresponding code merely returns lib/Foo/Bar.pm for t/Foo/Bar.t (and vice versa). It's trivial to write. However, the make_test_stub is what I added today. It's a hack (as many good experiments are!), but it does the following:

• Exit if we're not about to enter a .t test.
• Exit if the .t test exists
• Write a subtest for every public function

It looks like this:

use strict;
use warnings;
use autodie ':all';
use Class::Inspector;
use Sub::Information;

my ( $package_file, $test_file ) = @ARGV;

my $package = $package_file;
$package =~ s{\.pm$}{} or exit;
$package =~ s{^lib/}{};
$package =~ s{/}{::}g;

exit if -e $test_file;

eval "use $package";
die $@ if $@;

open my $fh, '>', $test_file;
print $fh <<"END";
use Test::Most;
use $package;

END

my $functions = Class::Inspector->function_refs($package);

my @functions;
foreach my $function (@$functions) {
    my $info = inspect($function);
    my $name = $info->name;
    next if $name =~ /^_/ or $name =~ /^[[:upper:]_]+$/;
    next if $info->package ne $package;
    push @functions => $name;
}

foreach my $function (@functions) {
    print $fh <<"END";
subtest "Verify $function" => sub {
    can_ok '$package', '$function';
};

END
}

print $fh "done_testing;\n";

So now, if I edit a file named lib/Weborama/Collect/Component/Publisher.pm and it has no tests, I hit ,gg in vim and get this:

use Test::Most;     
use Weborama::Collect::Component::Publisher;

subtest "Verify code" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'code';
};

subtest "Verify halt" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'halt';
};

subtest "Verify has_landing_url" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'has_landing_url';
};

subtest "Verify headers" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'headers';
};

subtest "Verify landing_url" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'landing_url';
};

subtest "Verify new" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'new';
};

subtest "Verify response" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'response';
};

done_testing;

This has a few benefits:

• You start out with working code rather than an empty test file.
• Even for modules that have no tests, you'll at least know that you can "use" the module.
• Even if you don't update the stub test, you'll get a test failure if someone removes part of the public API.
• It's harder to forget to test a particular public function.
• Continuing to evolve standards for testing can make our test suites easier to manage.

So I used this to quickly generate some stub tests for one of our projects and a few minutes later I had these extra tests in the test suite (I thought it prudent to obscure these test names):

$ git st --porcelain|grep ^A|cut -d' ' -f3|xargs prove -l
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxx.t.................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxx/xxxx/xxxxxxxx.t...... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxx/xxxx/xxxxxxxxxxxx.t.. ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxx.t......................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx.t......................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx/xxxx.t.................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx/xxxx/xxxxxxxxxx.t......... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx/xxxx/xxxxxxxxxxxxxx.t..... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx.t................ ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxx.t......... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxxx.t..... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxxxx.t.... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxx.t....... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxx/xxxxxxxx.t............ ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxxxx.t........................ ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxx.t............................. ok
All tests successful.
Files=16, Tests=53,  7 wallclock secs ( 0.06 usr  0.01 sys +  6.48 cusr  0.28 csys =  6.83 CPU)
Result: PASS

Naturally, not all tests can be shoe-horned into this one-size fits all methodology, but shortly after I recommended this at the BBC, devs noticed it was much easier to work with the test suite. At my current position, I'm finding that other projects have adopted this pattern and it makes switching between repositories and getting up to speed much easier.

This idea is just an experiment, but I'd love to hear feedback and suggestions.

If you liked this post, don't forget to check out my Beginning Perl book.