Tim King

The first part of this series described Test Hierarchy, a hierarchy of test classes that mirrors the classes under test, and explained why it’s an antipattern. Part two explored what makes a good unit test and why Test Hierarchy does not. This third and final post reflects on why programmers use Test Hierarchy and why these reasons aren’t persuasive.

I think there are a couple reasons why programmers use Test Hierarchy.

Test Hierarchy may appear to “just make sense” at first blush. After all, you have a hierarchy of classes under test—superclasses and subclasses—and you have a collection of test classes. It seems very symmetrical to have the test classes mirror the classes under test.

However, there’s no design justification for the test classes to be arranged in a parallel hierarchy. The easiest way to see this is to consider what happens when developers get tired of Test Hierarchy. What do they do? They drop back to procedural tests, with no inheritance at all. If you don’t need Test Hierarchy to test object-oriented code using procedural tests, why do you need it when using Test::Class? Answer: You don’t.

Test inheritance should only be used to meet the needs of the tests, not the needs of the code under test.

This usually means that if we have test superclasses, they specifically contain shared setup and teardown code or test utility functions. See the Testcase Superclass pattern, by which a test class can inherit common functionality from an abstract test superclass. Using this pattern, the test hierarchy is organized in order to share common code across the entire project’s tests or an entire subsystem’s tests.

(But use SharedTestModule qw(shared_function) is still preferred over inheritance, because it more explicitly states what is being shared and where.)

I’ve also seen programmers appeal to the Liskov Substitution Principle. This is the idea that if Bat is a subclass of Mammal, then any code that requires a Mammal can be handed a Bat without ill effects. Barbara Liskov and Jeanette Wing formally defined it like this:

Let ϕ(x) be a property provable about objects x of type T. Then ϕ(y) should be true for objects y of type S where S is a subtype of T. (“Behavioural Subtyping Using Invariants and Constraints.” Barbara Liskov, Jeanette Wing. CMU-CS-99-156. MIT Lab, July 1999.)

In other words, a subclass adheres to the same interface contract as its superclasses.

Some programmers will say that if Bat is a subclass of Mammal, then a Bat can do anything that a Mammal can. In other words, a Bat “is a” Mammal. Therefore, it directly follows that BatTest should test all the Mammal behaviors that Bat inherits.

No, it doesn’t, and no, it shouldn’t.

To understand why, consider a couple simple cases.

• If a Mammal has a care_for_young() method, then every Bat must also be able to care_for_young(). This does not mean that the way bats care for their young is exactly the same as every other mammal. In fact, it’s distinctly different, because baby bats have needs that are distinct from the needs of other baby mammals. In fact, Mammal::care_for_young() may even be an abstract method that dies with a “must be implemented in subclass” error.

• Similarly, every Animal can move(). That means every Mammal also can move(), because Mammal is a subclass of Animal. By extension every Bat can also move(), because Bat is a subclass of Mammal. Now explain to me how the way a bat moves is identical to the way a sloth moves or the way a tarantula moves. It isn’t.

A subclass adheres to the same interface contract as its superclasses. It does not necessarily implement identical behaviors.

Therefore, just because a Bat “is a” Mammal, that doesn’t mean that a BatTest “is a” MammalTest. Actually, no, a BatTest is not a MammalTest. Not even close. Both BatTest and MammalTest are just tests. Or they might, at most, be derived from OrganismTest abstract class which contains helper methods to set up and manage test fixtures common to all organisms.

Test::Class can test anything straight Test::More can, and vice-versa. The power in Test::Class is not as legend says in testing object-oriented code. The power Test::Class brings is its ability to collect related test methods together, run them independently, and inherit setup and teardown. (I’ll explore more of these details in Testing Strategies for Modern Perl.) Each test class should be derived directly from Test::Class or from an abstract subclass thereof. Never inherit test methods. Just don’t do it.

Peace, love, and may all your TAP output turn green…

This post originally appeared on The Perl Shop blog as “Why Programmers Use the Test Hierarchy Antipattern.”