Ovid

If you use Test::Class frequently, sooner or later you'll see the following:

sub startup : Tests(startup => 1) {
    my $test = shift;
    my $engines = $test->start_engines;
    ok $engines->are_running, 'Engines should start OK.';
}

At first blush that looks sensible, but I don't think it is. I'll explain my view of the problem and hope that others can explain why I might be wrong.

There are, in my opinion, two reasons not to use that construct. First, why is that a test? If that test fails, the rest of the tests in that class won't be run, but in this case, why not just die? Test control methods like startup, shutdown, setup and teardown shouldn't be an extra place to stuff tests, they're places to control the actual tests. If something has gone horribly wrong in them, just die and the rest of the tests in that class will be skipped.

Now that doesn't sound to everyone like a compelling argument, but what happens when you subclass?

sub startup : Tests(startup => 1) {
    my $test = shift;
    $test->next::method;
    ok $test->create_grue, 'Grue created';
}

Oops. Now you have a problem. That "next::method" has run a test in your superclass and you've run two tests instead of one (a patch to Test::Class could fix this, but you would likely break existing tests which workaround this issue). This means that you now have to count the tests in your superclass(es) startup method(s):

sub startup : Tests(startup => 2) {
    my $test = shift;
    $test->next::method;
    ok $test->create_grue, 'Grue created';
}

That looks wrong because you only see one test. Then when someone adds a test in a superclass startup, they've broken subclasses. Since there's no real reason (that I can see) to have tests in these test control methods and since they can be very fragile in the face of inheritance, why have them? I recommend that people don't use them and I've eliminated them in our production code at the BBC with no drawback. If something fails, just die and let Test::Class do the right thing.

Update: I completely forgot that you can use the following syntax to avoid counting superclass tests:

sub startup : Tests(startup => +1) {
    my $test = shift;
    $test->next::method;
    ok $test->create_grue, 'Grue created';
}

Note that you're specifying the additional number of tests, not the absolute number of tests. This renders much of my argument moot.