It's very interesting watching people use Test::Class. Usually, it's wrong. This isn't really a fault of Test::Class, though (well, not much), but rather, it's a fault of its documentation. It needs more.

Why?

First off, why should you use Test::Class?

• It's a natural fit for testing OO code.
• It makes it easy to control your state for every test (more on this).
• You're running Catatlyst.

Load times will kill you

If you're running Catalyst, particularly with a lot of Moose and DBIx::Class goodness, you'll likely find that your startup time suffers. If you use something like Test::Class::Load (bundled with Test::Class), you can load and run your tests in a single process. Why is this useful?

veure $ time perl -MVeure -e 1

real 0m1.984s

veure $ find t/lib/Tests/For/ -name *pm | wc -l
          17

With roughly two seconds to load Veure, if those 17 test classes were old-fashioned t/*.t files, I'd be adding more than half a minute to a test suite which currently takes just under half a minute to run. In other words, I'd be doubling my test time. For the code base at work, it loads in roughly 5 seconds. With the number of test classes we have, this would add almost 20 minutes to a test suite which takes an hour to run. Not good. See yesterday's blog post to understand why this is bad.

Why aren't people using Test::Class

So if it saves so much time when you run tests, why aren't more people using it? Well, I generally see two reasons. First is not understanding the importance of state. Second is forgetting basic OO. It doesn't quite apply the same way for test classes, but it's still important.

State

Not understanding state breaks a number of Test::Class suites that I see. Because test methods unfortunately run in alphabetical order, you can have tests very sensitive on the order in which they're run. Test "aaa" runs before "bbb" and leaves that customer record in the database that "bbb" depends on . Run "bbb" without "aaa" and bbbOOM!

To deal with this in Veure, my setup method guarantees a pristine environment. Regrettably, this slows my test suite down a bit and I'm only getting about 12 tests a second run due to rebuilding time, but it also means I get far greater isolation in my individual tests and thus, greater confidence (it also means that 10,000 tests will run in under 14 minutes, if I can maintain this speed. That's faster than most enterprise test suites I run across).

To repeat this: every test method must have a known state to start with. Isolation is important.

OO

These are classes, folks. If you subclass, don't forget that you can override methods. Why is this important? Look at some suites written with this and tell me if you see the following:

# must be run first!
sub aaa_setup : Tests(setup) { ... }

# (in a subclass)
sub setup : Tests(setup) { ... }

Why is that done? Because these are run in alphabetical order and you want to guarantee that your parent class setup runs before the child class setup. And then someone inserts another layer in the hierarchy and you see ccc_setup along with a comment to investigate the setup methods to make sure they're alphabetized correctly.

These are methods. They do semantically identical things, so here's how you guarantee correct execution order:

sub setup : Tests(setup) {
    my $test = shift;
    $test->next::method;
    # more setup here
}

Alphabetization goes away with this. It does mean that you don't want to put tests in the setup (or startup, shutdown or teardown methods) due to difficulty in tracking test counts, but really, you don't want to do that anyway. Have those methods die or bail out if your environment is set up incorrectly. It's much simpler that way.

Next, don't do this:

package My::Test::Class::Customer;

use Test::Class::Most parent => 'My::Test::Class';

my $URL_BASE = 'http://www.example.com';

sub foo : Tests {
    my $test = shift;
    my $mech = $test->mech;
    $mech->get_ok("$URL_BASE/some/path", "We can get /some/path");
}

I confess that I have been lazy in the past and done stuff like this. It's a bad lazy. Not only do subclasses not have access to that variable, but it leads to declaration timing problems. Here's one way to rewrite this:

package My::Test::Class::Customer;

use Test::Class::Most parent => 'My::Test::Class';

sub startup : Tests(startup) {
    my $test = shift;

    # my base class always has empty test control methods like "startup"
    # to guarantee that I never have to worry about whether or not this is safe
    $test->next::method;
    $test->{_url_base} ='http://www.example.com';
}
sub url_base { $_[0]->{_url_base} }

sub foo : Tests {
    my $test     = shift;
    my $mech     = $test->mech;
    my $url_base = $test->url_base;

    $mech->get_ok("$url_base/some/path", "We can get /some/path");
}

Need to override that in a subclass? No problem. Worried about the declaration assignment timing problem? It goes away. Your code is clean and things magically work the way they're supposed to.

Enhancements for Test::Class

It would be nice if Test::Class offered simple accessor generation to store properties. Then people would be less tempted to declare variables at the top of the code. It would also be useful to have randomized test order (and printing a see if tests fail, thus allowing you to rerun 'em in the same order).

Test::Class is still a fantastic module and well-thought out. We just need more docs for it.

(I really should put together a Test::Class class)