Tim King

Test::Class is particularly good at testing object-oriented code, or so it is said. You can create a hierarchy of test classes that mirrors the hierarchy of classes under test. But this pattern, common in Perl projects, is conspicuously missing from the rest of the xUnit world, and with good reason.

---

We've all heard of it.

Our project has a class Animal that implements method move() (because all animals can move). This class has a test class AnimalTest that derives from Test::Class and has a test for $animal->move().

So far so good.

Our project also has a class Bat that derives from Mammal, which derives from Animal, and implements method fly(). So we create a test class BatTest that derives from MammalTest, which in turn derives from AnimalTest, and has a test for $bat->fly(). That means that BatTest not only exercises all the behavior of Bat but also of Mammal and Animal, because it inherits all the tests in its ancestors.

Wow! What a cool feature! We get all that testing functionality essentially for free by inheritance!

And if the foregoing description sounded confusing, just imagine how good it's going to get as we extend the object hierarchy.

Repeat for umpteen different classes.

This arrangement of test classes is what I mean by Test::Class Hierarchy, or more generally, Test Hierarchy.

Multiple prominent sources in the Perl community recommend Test Hierarchy.

• This advice appears in Perl Testing: A Developer's Notebook, by Ian Langworth and chromatic (published 2005).


• Curtis "Ovid" Poe recommends it on the Modern Perl site (2009), and then reiterates that advice in Beginning Perl (published 2012).

And most notably, when I ask a roomful of Perl developers about their experiences with Test::Class, I'm sure to hear at least one person complain about "the rabbit hole of inheritance," as jnap once put it in a conversation. Of course, not every project misuses Test::Class and its support for inheritance, but as he noted, "I've just seen it so wildly abused." (And to be fair, this is not the only abuse of Test::Class, but it's the pattern I'm examining at the moment.)

• This makes fragile, overlapping tests. When we inherit test methods in this way, we end up with test action at a distance. That is, each test class includes tests that are defined in its superclasses, which are completely different modules. A change to any of the superclasses can produce failures in any and all of the subclass tests.


• These tests are obscure. We can't know by looking at the test module what functionality it's testing, at least not without following the inheritance hierarchy all the way to the top.


• And they're slow to boot. The test suite takes exponentially longer to run than it needs to, because the same tests are being run over and over again in each subclass.

In the words of Alyssa Mastromonaco (or maybe her publisher): Who thought this was a good idea?

Enough Perl projects use Test Hierarchy that it pops up in criticisms of Test::Class itself, and the negative effects are a significant point when they do.

It bears noting, however, that this practice is strongly discouraged in the rest of the programming world. It's so rare, in fact, that Gerard Meszaros doesn't even mention it in his book xUnit Test Patterns.

Rather, the recommended practice is to inherit our test classes directly from Test::Class (or possibly from a project-or subsystem-specific test base class—but that's a different post). In general, we use as little hierarchy as possible, and whatever hierarchy we do use is organized according to the needs of the tests, not the needs of the system under test. And we never inherit test methods (although we may inherit setup and teardown code).

In summary, we write independent test classes, and we never inherit test methods.

That qualifies Test Hierarchy as a Perl antipattern:

1. It's a commonly used structure that despite initially appearing to be appropriate and effective, has more bad consequences than good; and


2. Another solution exists that is documented, repeatable, and proven to be effective.

In the next post, I talk about why Test Hierarchy does not even make good unit tests.

Peace, love, and may all your TAP output turn green…

---

This post originally appeared on The Perl Shop blog as "Test::Class Hierarchy Is an Antipattern."