For those who are very familiar with using Test::Class (or if you've read my Test::Class tutorial), you may be used to seeing a base class that looks like this:

package My::Test::Class;
use parent 'Test::Class';

INIT { Test::Class->runtests }

sub startup  : Tests(startup)  {}
sub setup    : Tests(setup)    {}
sub teardown : Tests(teardown) {}
sub shutdown : Tests(shutdown) {}

1;

The empty test control methods are there so that a subclass knows it can always safely do this:

sub startup : Tests(startup) {
    my $test = shift;
    $test->next::method;
    # more code here
}

Those stub test methods are no longer needed with Test::Class::Moose, but until today, that INIT block was an annoying code smell that had some unfortunate side-effects. Let's make that go away.

First of all, what does that INIT block do?

INIT { Test::Class->runtests }

In Test::Class, that would simply run all loaded test classes at INIT time. However, that means you could do this:

prove -lv t/lib/path/to/my/test/class.pm

And voilà, you test class could be executed just like it's a *.t program. That's because it would be loaded, it would inherit from your base class, the INIT block would fire and Rube Goldberg would be laughing in his grave.

Sadly, this leads to nasty bugs like this one I blogged about almost 3 years ago. Basically, if you use $some::module, the code in that module is executed before INIT fires but the subroutines are not (aside from import()). However, if you load the code directly with, say perl t/lib/path/to/my/test/class.pm, the INIT block fires before the code in the module is executed. You can read perldoc perlmod for more information about INIT and friends.

How does this impact Moose?

At first glance, you might not think issues with INIT and friends would impact Moose, but there are a couple of subtle repercussions. For example, we know that subroutine attributes only fire at compile time. That's why the Moose manual explains how to use inherited subroutine attributes with Moose. In short, you have to ensure that you're inheriting at compile-time, not at CHECK/INIT/UNITCHECK or runtime:

BEGIN { extends 'Some::Class' }

Since Moose isn't part of the Perl language and much of its behavior fires at runtime, you can have some strange issues, such as when I wrote the following code:

package TestsFor::Person::Employee;
use Test::Class::Moose extends => 'TestsFor::Person';

sub extra_constructor_args {
    return ( employee_number => 666 );
}

BEGIN {
    after 'test_constructor' => sub {
        my $test = shift;
        is $test->test_person->employee_number, 666,
          '... and we should get the correct employee number';
    };
}

1;

That's really ugly and, in fact, I received an email from someone who called me on that, arguing that Test::Routine and Test::Roo were simpler in this case. I think he was right and I thought for a while about the best way to resolve this. You shouldn't have to think about timing issues like this when you're writing your test classes.

A Cleaner Test::Class::Moose

Today I've released Test::Class::Moose version 0.11 and it makes this case easier to handle. Your base class now can look like this:

package My::Test::Class;
use Test::Class::Moose;
1;

In fact, you may not even need a base test class at all — just have your classes use Test::Class::Moose directly — but a base test class is handy enough that people put all sorts of interesting code in there.

So how do you do the INIT trick that lets you run an individual test class? I now recommend use of a driver *.t script for your test suite. Here's what one might look like:

use Test::Class::Moose::Load 't/lib';
my $test_suite = Test::Class::Moose->new(
     show_timing  => 1,
     randomize    => 0,
     statistics   => 1,
     test_classes => $ENV{TEST_CLASS},
)->runtests;

my $report = $test_suite->test_report;
# the reporting object, of course, can be ignored if you
# don't need reporting on the test suite

Unlike Test::Class, Test::Class::Moose has a constructor that lets you control the test suite behavior prior to running the suite. There is a new attribute, test_classes which takes the name of a test class or an array reference of test class names. Only those test classes will be run. Yes, you could have written a Test::Class::Moose subclass to control this, but I wanted to make this built-in because running individual classes is a common use case. Now, instead of this:

prove -lv t/lib/path/to/my/test/class.pm

You do this:

TEST_CLASS=Class::I::Want::To::Run prove -lv t/test_class_tests.t

And things work just like the are supposed to. As a bonus, that nasty BEGIN block in my TestsFor::Person::Employee class goes away:

package TestsFor::Person::Employee;
use Test::Class::Moose extends => 'TestsFor::Person';

sub extra_constructor_args {
    return ( employee_number => 666 );
}

after 'test_constructor' => sub {
    my $test = shift;
    is $test->test_person->employee_number, 666,
      '... and we should get the correct employee number';
};

1;

And your test classes now look like proper Moose code rather than worrying about BEGIN versus INIT phases.

I'll be giving a talk about Test::Class::Moose at YAPC::NA and I've also proposed it for YAPC::EU. I hope to see you there!

So I've finally gotten around to updating Test::Class::Moose to have tags. You can read my previous post when I explain why they're useful. You can go out to github and grab it now, or wait a bit for it to hit your favorite CPAN mirror.

The constructor is very straightforward. For the case I previously described when the network went down? Skip test methods with a network tag!

Test::Class::Moose->new(
    exclude_tags => 'network', # scalar or arrayref of tags
)->runtests;

Or maybe you want to run your Big Data and API tests, but skip deprecated methods:

Test::Class::Moose->new(
    include_tags => [qw/bigdata api/],
    exclude_tags => 'deprecated',
)->runtests;

This should solve a common issue where people want to attach metadata to their test suite but there's no clean support for it.

Unfortunately, this was implemented with Sub::Attribute. That's a fantastic module, but it's an optional dependency that some don't want. I had to use that module, though, because there's a little-known edge case where subroutine attributes won't fire if the module was loaded via "eval". I might try another swing at fixing this in the future (via require and a path in Test::Class::Moose::Load?), but for now, if you can't load Sub::Attribute, a warning will be issued and your tag filters will be ignored.

Patches welcome for improving this feature (there are many things which can be done here), but for now, this is the last major piece really needed to make this the goto module for many large test suites.

And that's really the most interesting idea of this entire post: your customer's behavior is more important than your application's behavior.

However, if you're trying to build a business (and I'm focusing on that in this post), then it should be blindingly obvious that customers are more important than code, but that assertion leaves many people scratching their heads. Reddit, for example, was underwhelmed by the post, but one commenter (the only one on Reddit, honestly), reasoned through what I described and hit the nail squarely on the head:

And if you're relying on watching user behaviors to detect a noticeable but insignificant bug, there are many you're going to miss. And if you're relying on a production rollback to fix those bugs, you're going to let many of them through because it's less costly than a rollback. And eventually your program is going to be a big steaming mess that nobody likes, but that ill will can't be traced back to a particular "unexpected behavior" but rather to the confluence of all of them.

I loved that comment because it divined a very real problem with the approach I described.

In my "no testing" experience, your code, while not always turning into a big ball of mud, at minimum turns into an unweildy pile of ones and zeroes that isn't very fun to work on. I've worked at many different companies and I've never seen a well-designed, easy-to-hack system that doesn't have tests. It simply never happens (in my experience). As a result, you get a system that's harder to extend, harder to refactor, has more bugs and, crucially for many companies, less attractive for developers to work on. I still remember one company in London that had acquired this reputation and while they paid a reasonable salary, no one wanted to work there (London.pm members might guess who this is, but I won't confirm it and and I hope you won't post that here).

So you can be agile without testing, but you are going to accumulate technical debt and you have to be willing to service that debt. But if you want to minimize your technical debt, when must you test? That's simple: harm.

When people argue about whether or not there is a universal moral standard, the only one the really agree on is "do no harm" (though there's little agreement about what constitutes harm). For me, from an ethical standpoint, I have to argue in favor of do no harm in regards to software testing. In fact, it's not just a passive "do no harm", it's an active "mitigate harm" that I think is the only ethical choice. If you're not going to write tests to verify that the thumbnail size is correct, that's your choice. If you're not going to write tests verifying that you don't double-charge a customer who accidentally double-clicks "Submit", then I have to say that you've taken an unethical decision. I might want to write a lot more tests, but the only tests I'm going to insist must be written are those which prevent actual harm, even though I might want a lot more tests written.

So those are the tests you must write. What about the tests you should write? If you don't follow the monitoring approach that I described, then I would argue that you should write tests for anything and everything and hope that your customers like what you do. Note that this is actually what many "agile" shops do, so in this regard, I'm not proposing anything radical. However, if you do follow the monitoring approach, I would argue that at the very least you should write integration tests catching fatal errors. A customer still might buy something if they only see one columns of products instead of two, but they won't buy anything if the application crashes (not writing tests for this area is exposing your company to harm, but at least it's a choice).

I would also argue that if you have actions that customers must take for the success of your business, write enough integration tests to verify that your customers can take said action. Have a Web site? Make sure you know your conversion funnels and that people can get through them. Have an iPhone app? Make sure that the in-game extras people can buy actually show up and can be purchased.

Note that this approach, again, focuses on customers and not developers. You must not bring harm to your customers and you must always provide a way for your customers to complete the actions you wish them to complete. It's also good if you have logging to verify the most common actions your users take so you can at least verify that those actions aren't fatal: the Pareto rule is very much your friend here.

Over time, you'll wind up with enough integration tests that devs can still refactor with a modicum of safety and, if it's a large enough refactor, it's easy to add extra tests to verify that the behavior you're targeting is successful.

There is also a curious side-benefit to this: test coverage might be more useful. This sounds paradoxical, but remember, this approach focuses on whether or not your customers take the actions you need for your business to be successful — not whether or not your application does what you think it's supposed to do. As a result, your tests are more focused on the bulk of your business needs instead of that "Facebook Like" button and, if tests are managed correctly, your code coverage could show that there are large parts of your system that you're not actually using. Remember: for must of us, a unit test on a function will show that function as being tested, regardless of whether or not it's dead code. However, it's a lot harder to get coverage of dead code when you focus your tests on what your customers are actually doing.

To be fair, I generally don't test this way because very few companies have strong enough monitoring or deployment procedures for this to work. It's very much a "cutting edge" approach and even if you can get your systems evolved enough to allow this, my experience is that many developers are going to struggle with the idea that customer behavior is more important than application behavior.

Instead, I'll leave you with this quote from the Reddit thread I referred to:

I think your overall message is, "don't succumb to cargo-cult programming practices"--which is a great one! But the structure of your post sounds like you're suggesting doing user-monitoring in place of traditional testing, while you're really suggesting doing user-monitoring in addition to traditional testing, employed strategically.

Clearly I need to hire that person to be my editor.

TL;DR

If you want to try a radically different agile strategy:

• Use continuous deployment
• Monitor any behaviors that absolutely impact the bottom line
• Write integration tests to catch all fatal errors
• Write integration tests to validate conversion funnels
• Write tests to prevent any "harmful" behavior

Instead of repeating mantras, trying focusing on your customers instead of the code. I've seen this in action (minus some of the testing I recommend) and it works astonishingly well. Which would you rather have: code that does what you want or customers that do what you want? It's your call.

See also: Code evolution versus intelligent design.

sub test_get_profile : Tags(redis network) {
    ... tests go here
}

What could you do with that? Well, the other day our network went down at work. My test suite blocked because all of the tests which required a network would hang. With the above, I could do this:

Test::Class::Moose->new( exclude => 'network' )->runtests;

And have carried on with my work.

Or maybe you have tests that you know could run in parallel:

sub test_some_unit_test : Tags(parallel) {
    ... tests go here
}

With that, if someone writes a forking version of Test::Class::Moose, you could automatically pick out all of the tests which can safely run in parallel and the tests which cannot would be run sequentially in another child.

Of course, if you parallelize the tests, it would be nice if you knew how long different test classes and methods ran: it would make it easier to write a scheduler. Oh, that's right, Test::Class::Moose has reporting (and a branch where I'm beefing up the reporting docs).

There's a lot of potential here. I'm currently working on improving the documentation, but I think the abstract should say "test suites for power users". With all of the features it has, it's shaping up into a pretty nifty tool.

If you like this idea, I've added it to play.perl.org.

1. named your Redis connection
2. issued the client-list command
3. found the connection with the correct name

4. and written a parser for this:

   addr=127.0.0.1:36188 fd=5 name= age=9 idle=0 flags=N db=2 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client

Oops. We forgot to name our connection, so we can't uniquely identify this one.

Instead, of doing things the "right" way, I decided that YAGNI applied. In a subclass of Redis, I was trying to write the following code because it seemed simpler:

has 'database' => (
    is      => 'rw',
    isa     => 'Int',
    default => 0,
);

after 'select' => sub {
    my ( $self, $database ) = @_;
    $self->database($database);
};

That failed. Then I remembered that the excellent Redis module by Pedro Melo uses AUTOLOAD for the Redis command. I can't apply a method modifier to a method that does not yet exist.

I honestly can't fault him for this. It means that when a new version of Redis is released, the code just magically works.

So I tried this:

sub select {
    my ( $self, $database ) = @_;
    $self->next::method($database);
    $self->database($database);
}

Nope. Same issue. I finally relied on SUPER::select. That worked fine, though it limits my choices due to the well-known SUPER bug (SUPER::$method calls the parent of the class the code is compiled into, not the parent of the invocant).

The following shows the issue:

use Modern::Perl;                                                                                             
{
    package Parent;

    sub AUTOLOAD {
        our $AUTOLOAD;
        my $command = $AUTOLOAD;
        $command =~ s/.*://;
        return "command was $command";
    }
}
{
    package Child;
    our @ISA = 'Parent';

    sub new { bless {} => shift }
    sub foo { return shift->SUPER::foo }
    sub bar { return shift->next::method }
}
my $child = Child->new;
say $child->foo;
say $child->bar;

And the output:

command was foo
No next::method 'bar' found for Child at ...

This was pretty easy for me to find and fix, but I can easily imagine for many who are new to Perl, this would have been completely mystifying.

Here's how diag() all test classes from slowest to fastest, with time information:

use Test::Most;
use Test::Class::Moose::Load 't/lib';    # assumes your T:C:M tests are here

my $test_suite = Test::Class::Moose->new;
$test_suite->runtests;

my $report = $test_suite->test_reporting;

# sort from slowest to fastest
my @classes =
  sort { $b->time->real <=> $a->time->real } $report->all_test_classes;
foreach my $class (@classes) {
    my $name = $class->name;
    my $duration = $class->time->duration; # a human-readable version
    diag "$name runtime: $duration";                                                                                                       
}

You can do that at a test method level, too. Plus, the time is broken down into system, user, and real time.

By now you may have heard of Test::Class::Moose. I wrote this to solve a need that many people have: they want the awesomeness of Test::Class, but the modern OO facilities of Moose.

Test::Class::Moose isn't for testing Moose classes, it's for testing anything you would have previously used Test::Class for, except that now you get Moosy (Moosee? Moosey?) goodness to go with it. I'll be attending YAPC::NA 2013 in Austin and I've pitched a Test::Class::Moose talk and, even if it doesn't get accepted, I figured I should at least write the slides. One of my most common uses cases (and and itch I always rescratch whenever I use Test::Class, Test::Class::Most or, now, Test::Class::Moose) caused one of my slides to have too much code.

So now it's fixed, released to the CPAN, and available for everyone to tell me it's "too magical" (a complaint I've heard in the past). Here's the problem and how I solved it.

package Person;
use Moose;

has [ 'first_name', 'last_name' ] => (
    is       => 'ro',
    isa      => 'Str',
    required => 1,
);

sub full_name {
    my $self = shift;
    return join ' ' => $self->first_name, $self->last_name;
}

1;

To use any of the Test::Class* modules, you start by writing a base class. Here's what ours looks like:

package My::Test::Class;

use Test::Class::Moose;

INIT {
    Test::Class::Moose->new(
        show_timing => 0,
        randomize   => 0,
        statistics  => 1,
    )->runtests;
}

1;

Note: you don't need to know Test::Class for this example, but if you want to know more, see my (out of date) Test::Class tutorial on Modern Perl Books.

And our actual test class might look like this:

package TestsFor::Person;
use Test::Class::Moose parent => 'My::Test::Class';
use Person;

sub test_constructor {
    my $test = shift;

    ok my $person = Person->new(
        first_name => 'Bob',
        last_name  => 'Dobbs',
    ), 'We should have a test person';

    isa_ok $person, 'Person', '... and the object it returns';
    is $person->full_name, 'Bob Dobbs',
      '... and it should return the correct full name';
}

1;

Running that with prove -lv t/lib/TestsFor/Person.pm generates the following output:

1..2
#
# Running tests for My::Test::Class
#
    1..0 # SKIP Skipping 'My::Test::Class': no test methods found
ok 1 # skip Skipping 'My::Test::Class': no test methods found
#
# Running tests for TestsFor::Person
#
    1..1
    # TestsFor::Person->test_constructor()
        ok 1 - We should be able to create a Person object
        ok 2 - ... and the object it returns isa Person
        ok 3 - ... and it should return the correct full name
        1..3
    ok 1 - test_constructor
ok 2 - TestsFor::Person
# Test classes:    2
# Test methods:    1
# Total tests run: 3
ok
All tests successful.
Files=1, Tests=2,  3 wallclock secs ( 0.03 usr  0.01 sys +  0.46 cusr  0.05 csys =  0.55 CPU)
Result: PASS

So everything looks good so far, but that contains the very common anti-pattern I see in test class tests. Do you see the problem? It's very subtle and we'll see it in a moment.

Let's imagine that we have an employee subclass of person:

package Person::Employee;

use Moose;
extends 'Person';

has 'employee_number' => (
    is       => 'ro',
    isa      => 'Str',
    required => 1,
);

1;

A stub test for that could look like this:

package TestsFor::Person::Employee;

use Test::Class::Moose extends => 'TestsFor::Person';
use Person::Employee;

1;

That will actually run tests because we inherit our test_constructor method from TestsFor::Person. Let's see what that looks like:

1..3
#
# Running tests for My::Test::Class
#
    1..0 # SKIP Skipping 'My::Test::Class': no test methods found
ok 1 # skip Skipping 'My::Test::Class': no test methods found
#
# Running tests for TestsFor::Person
#
    1..1
    # TestsFor::Person->test_constructor()
        ok 1 - We should have a test person
        ok 2 - ... and the object it returns isa Person
        ok 3 - ... and it should return the correct full name
        1..3
    ok 1 - test_constructor
ok 2 - TestsFor::Person
#
# Running tests for TestsFor::Person::Employee
#
    1..1
    # TestsFor::Person::Employee->test_constructor()
        ok 1 - We should have a test person
        ok 2 - ... and the object it returns isa Person
        ok 3 - ... and it should return the correct full name
        1..3
    ok 1 - test_constructor
ok 3 - TestsFor::Person::Employee
# Test classes:    3
# Test methods:    2
# Total tests run: 6

Notice how we've inherited the test_constructor test, but it's just a useless double test. We're not actually testing our new class because we hardcoded the Person class name in TestsFor::Person. In fact, I remember one time many years ago when my tests were passing but the package didn't even compile! It turns out that in my test class, I had forgotten to use the package I was testing and my inherited tests had the parent class name hard-coded. I decided to never make that mistake again. Now I have my test classes (usually) automatically load the classes they are going to test.

Annoyingly, automatically using the classes I test is the code I keep rewriting, so now I have created Test::Class::Moose::Role::AutoUse. This role strips the leading segment of the test class package name (TestsFor::, for our examples) and will use the resulting class name and make it available via the class_name() attribute.

Change the base class to look like this:

package My::Test::Class;

use Test::Class::Moose;
with 'Test::Class::Moose::Role::AutoUse';

INIT {
    Test::Class::Moose->new(
        show_timing => 0,
        randomize   => 0,
        statistics  => 1,
    )->runtests;
}

1;

And the TestsFor::Person class now looks like this:

package TestsFor::Person;
use Test::Class::Moose parent => 'My::Test::Class';

sub test_constructor {
    my $test = shift;

    my $class = $test->class_name;
    ok my $person = $class->new(
        first_name => 'Bob',
        last_name  => 'Dobbs',
      ),
      'We should have a test person';

    isa_ok $person, $class, '... and the object it returns';
    is $person->full_name, 'Bob Dobbs',
      '... and it should return the correct full name';
}

1;

Note that we no longer need to use Person and we no longer hard-code the class name.

The TestsFor::Person class tests pass, but TestsFor::Person::Employee now die a horrible death, telling us that that employee_number argument is required for the constructor., even though we've not had to change the code for that class.

In other words, you can no longer forget to use the class under test and, if you don't hard-code the class name, your test inheritance works properly. (Note that I don't often his this bug any more because I avoid inheritance, but I still like that I can automatically use the packages I am testing).

Rather than leave you hanging, here's one way of fixing this:

package TestsFor::Person;
use Test::Class::Moose parent => 'My::Test::Class';

BEGIN { has 'test_person' => ( is => 'rw', isa => 'Person' ); }

sub extra_constructor_args {}

sub test_setup {
    my $test = shift;
    $test->test_person( $test->class_name->new({
        first_name => 'Bob',
        last_name  => 'Dobbs',
        $test->extra_constructor_args,
    }));
}

sub test_constructor {
    my $test = shift;

    ok my $person = $test->test_person, 'We should have a test person';

    isa_ok $person, $test->class_name, '... and the object it returns';
    is $person->full_name, 'Bob Dobbs',
      '... and it should return the correct full name';
}

1;

A few notes about that. First, the test_person attribute is, well, an attribute. Thus, Test::Class::Moose knows that it is not a test method, even though it begins with test_ (all methods beginning with test_ are automatically test methods).

Second, it's wrapped in a BEGIN block due to a subtle bug that crops a few times. In this case, if Moose attributes (and as you'll see later, method modifiers) were built into Perl and fired at compile time, this bug would go away.

Finally, the test_setup method is a test control method which is called before every test method. It ensures that you always have a fresh sample instance of the object you're testing.

Then our subclass looks like this:

package TestsFor::Person::Employee;
use Test::Class::Moose extends => 'TestsFor::Person';

sub extra_constructor_args {
    return ( employee_number => 666 );
}

BEGIN {
    after 'test_constructor' => sub {
        my $test = shift;
        is $test->test_person->employee_number, 666,
          '... and we should get the correct employee number';
    };
}

1;

We have to wrap our method modifier in a BEGIN block due to the bug mentioned earlier. Aside from that, though, you'll note that we extend the parent constructor behavior to only test the new behavior. That helps us avoid duplicated test code.

And the test output:

1..3
#
# Running tests for My::Test::Class
#
    1..0 # SKIP Skipping 'My::Test::Class': no test methods found
ok 1 # skip Skipping 'My::Test::Class': no test methods found
#
# Running tests for TestsFor::Person
#
    1..1
    # TestsFor::Person->test_constructor()
        ok 1 - We should have a test person
        ok 2 - ... and the object it returns isa Person
        ok 3 - ... and it should return the correct full name
        1..3
    ok 1 - test_constructor
ok 2 - TestsFor::Person
#
# Running tests for TestsFor::Person::Employee
#
    1..1
    # TestsFor::Person::Employee->test_constructor()
        ok 1 - We should have a test person
        ok 2 - ... and the object it returns isa Person::Employee
        ok 3 - ... and it should return the correct full name
        ok 4 - ... and we should get the correct employee number
        1..4
    ok 1 - test_constructor
ok 3 - TestsFor::Person::Employee
# Test classes:    3
# Test methods:    2
# Total tests run: 7

Not everyone likes Test::Class and not everyone likes Moose, but if you like both and you want to harness the power of both, it's all there for you. I'd like to find a cleaner way around the BEGIN block issue, but this is a decent start.

Also, if you don't like how the class names are determined with Test::Class::Moose::Role::AutoUse, you can override the get_class_name_to_use method.

• Trying to figure out where the tests are
• Writing boilerplate
• Finding yet another package without tests

That last one is particularly vexing when you discover that your code is failing because another package doesn't load, but it doesn't have tests. So I fixed that.

Some of what follows is a repeat of things written in previous posts, but it's important enough that it bears repeating.

$ ls t/*t | wc -l
312

And you're sitting there weeping because the nice, orderly code in your lib/ directory is tested by a pile 'o junk. Other projects have subdirectories, but those subdirectories often group things by behavior, not package; your tests for lib/Foo/Bar/Baz.pm wind up in t/common/exceptions/, t/common/standards/, t/database/, t/miscellaneous/crap/tests/. Even if you memorize all of those subdirectories, you find yourself adding code that can, arguably, be in multiple subdirectories.

My recommendation was that our modules have one corresponding .t file. The agreement reached was that a module like lib/Foo/Bar.pm would go into t/Foo/Bar.t. I also have the following in my .vimrc:

noremap ,gg :call GotoCorresponding(expand("%"))<cr>                                                                                     

function! GotoCorresponding(module)                                                                                                       
    let file = system("get_corresponding ".a:module)
    if !empty(file)
        let ignore = system("perl /home/cpoe/bin/make_test_stub ".a:module." ".file)
        execute "edit " . file
    else
        echoerr("Cannot find corresponding file for: ".a:module)
    endif
endfunction

The get_corresponding code merely returns lib/Foo/Bar.pm for t/Foo/Bar.t (and vice versa). It's trivial to write. However, the make_test_stub is what I added today. It's a hack (as many good experiments are!), but it does the following:

• Exit if we're not about to enter a .t test.
• Exit if the .t test exists
• Write a subtest for every public function

It looks like this:

use strict;
use warnings;
use autodie ':all';
use Class::Inspector;
use Sub::Information;

my ( $package_file, $test_file ) = @ARGV;

my $package = $package_file;
$package =~ s{\.pm$}{} or exit;
$package =~ s{^lib/}{};
$package =~ s{/}{::}g;

exit if -e $test_file;

eval "use $package";
die $@ if $@;

open my $fh, '>', $test_file;
print $fh <<"END";
use Test::Most;
use $package;

END

my $functions = Class::Inspector->function_refs($package);

my @functions;
foreach my $function (@$functions) {
    my $info = inspect($function);
    my $name = $info->name;
    next if $name =~ /^_/ or $name =~ /^[[:upper:]_]+$/;
    next if $info->package ne $package;
    push @functions => $name;
}

foreach my $function (@functions) {
    print $fh <<"END";
subtest "Verify $function" => sub {
    can_ok '$package', '$function';
};

END
}

print $fh "done_testing;\n";

So now, if I edit a file named lib/Weborama/Collect/Component/Publisher.pm and it has no tests, I hit ,gg in vim and get this:

use Test::Most;     
use Weborama::Collect::Component::Publisher;

subtest "Verify code" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'code';
};

subtest "Verify halt" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'halt';
};

subtest "Verify has_landing_url" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'has_landing_url';
};

subtest "Verify headers" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'headers';
};

subtest "Verify landing_url" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'landing_url';
};

subtest "Verify new" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'new';
};

subtest "Verify response" => sub {
    can_ok 'Weborama::Collect::Component::Publisher', 'response';
};

done_testing;

This has a few benefits:

• You start out with working code rather than an empty test file.
• Even for modules that have no tests, you'll at least know that you can "use" the module.
• Even if you don't update the stub test, you'll get a test failure if someone removes part of the public API.
• It's harder to forget to test a particular public function.
• Continuing to evolve standards for testing can make our test suites easier to manage.

So I used this to quickly generate some stub tests for one of our projects and a few minutes later I had these extra tests in the test suite (I thought it prudent to obscure these test names):

$ git st --porcelain|grep ^A|cut -d' ' -f3|xargs prove -l
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxx.t.................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxx/xxxx/xxxxxxxx.t...... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxx/xxxx/xxxxxxxxxxxx.t.. ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxx/xxxx.t......................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx.t......................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx/xxxx.t.................... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx/xxxx/xxxxxxxxxx.t......... ok
t/xxxxxxxx/xxxxxxx/xxxxxxxxxxxxxx/xxxx/xxxxxxxxxxxxxx.t..... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx.t................ ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxx.t......... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxxx.t..... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxxxxx.t.... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxxxxxxx/xxxxxxxx.t....... ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxx/xxxx/xxxxxxxx.t............ ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxxxxxxx.t........................ ok
t/xxxxxxxx/xxxxxxx/xxxxx/xxxx.t............................. ok
All tests successful.
Files=16, Tests=53,  7 wallclock secs ( 0.06 usr  0.01 sys +  6.48 cusr  0.28 csys =  6.83 CPU)
Result: PASS

Naturally, not all tests can be shoe-horned into this one-size fits all methodology, but shortly after I recommended this at the BBC, devs noticed it was much easier to work with the test suite. At my current position, I'm finding that other projects have adopted this pattern and it makes switching between repositories and getting up to speed much easier.

This idea is just an experiment, but I'd love to hear feedback and suggestions.

If you liked this post, don't forget to check out my Beginning Perl book.