Test Program Names

Schwern recently wrote about not using numbers in test names. He's right and it's something I've been guilty of in the past, but I want to recommend that people go further and start naming test programs after packages. For example, with SQL::Statement, you have the following tests:

00error.t
01prepare.t
02executeDirect.t
03executeDBD.t
04names.t
05create.t
06group.t
07case.t
08join.t
09ops.t
10limit.t
11functions.t
12eval.t
13call.t
14allcols.t
15naturaljoins.t
16morejoins.t
17quoting.t
18bigjoin.t
19idents.t
20pod.t
21pod_coverage.t

So, if you want to write/find tests for the &SQL::Parser::feature subroutine, which .t file do you look in? Here's how I would name the tests:

t/sql/dialects/ansi.t
t/sql/dialects/anydata.t
t/sql/dialects/csv.t
t/sql/eval.t
t/sql/parser.t
t/sql/statement.t
t/sql/statement/function.t
t/sql/statement/functions.t
t/sql/statement/getinfo.t
t/sql/statement/operation.t
t/sql/statement/placeholder.t
t/sql/statement/ram.t
t/sql/statement/term.t
t/sql/statement/termfactory.t
t/sql/statement/util.t

Now it's immediately obvious that the tests for that would be in t/sql/parser.t.

If you need finer grained .t files, break them down behaviorally:

t/sql/parser-api.t
t/sql/parser-internal.t
t/sql/parser-exceptions.t

Or:

t/sql/statement/function-equal.t
t/sql/statement/function-noequal.t
t/sql/stat ement/function-lower.t

Not only are your tests easy to find, it's also a piece of cake to write editor tools to jump between tests and their packages. Plus, for people new to the project, it's easy to learn. Whenever I work on a large project, I usually find the test suite to have some cryptic organization which only the maintainers know and I have to grep or ack through the tests and guess. With this system, it's quick and easy to learn.

4 Comments

I like that idea a lot for unit tests, but I think it doesn't address testing coordinated behaviors that span multiple units. Still, moving away from numbered flat tests to meaningful names in a hierarchy makes lots of sense. There's no reason not to name behavior tests in a similar way.

All your files, not just the test ones, should have meaningful names. And sometimes, it makes sense to do the simple tests before the complex ones. And without leading numbers, the tests would be done alphabetically, which would mean you have to come up with names for the simple tests that are alphabetically before the complex ones. Which means you have to throw your meaningful names out the window. The best solution, so far, is to have leading numbers and meaningful names.

This is why I like Test::Class (and recently, Test::Sweet). Class-based tests make it very easy to organize tests. I know that Test::Foo::Bar exercises the methods in Foo::Bar. And T::F::B can inherit tests from its parent, which parallels the parent of F::B.

With Test::Sweet, you can even compose test classes with both inheritance and roles.

Leave a comment

About Ovid

user-pic Have Perl; Will Travel. Freelance Perl/Testing/Agile consultant. Photo by http://www.circle23.com/. Warning: that site is not safe for work. The photographer is a good friend of mine, though, and it's appropriate to credit his work.