Test Program Names
Schwern recently wrote about not using numbers in test names. He's right and it's something I've been guilty of in the past, but I want to recommend that people go further and start naming test programs after packages. For example, with SQL::Statement, you have the following tests:
00error.t 01prepare.t 02executeDirect.t 03executeDBD.t 04names.t 05create.t 06group.t 07case.t 08join.t 09ops.t 10limit.t 11functions.t 12eval.t 13call.t 14allcols.t 15naturaljoins.t 16morejoins.t 17quoting.t 18bigjoin.t 19idents.t 20pod.t 21pod_coverage.t
So, if you want to write/find tests for the &SQL::Parser::feature subroutine, which .t file do you look in? Here's how I would name the tests:
t/sql/dialects/ansi.t t/sql/dialects/anydata.t t/sql/dialects/csv.t t/sql/eval.t t/sql/parser.t t/sql/statement.t t/sql/statement/function.t t/sql/statement/functions.t t/sql/statement/getinfo.t t/sql/statement/operation.t t/sql/statement/placeholder.t t/sql/statement/ram.t t/sql/statement/term.t t/sql/statement/termfactory.t t/sql/statement/util.t
Now it's immediately obvious that the tests for that would be in t/sql/parser.t.
If you need finer grained .t files, break them down behaviorally:
t/sql/parser-api.t t/sql/parser-internal.t t/sql/parser-exceptions.t
Or:
t/sql/statement/function-equal.t t/sql/statement/function-noequal.t t/sql/stat ement/function-lower.t
Not only are your tests easy to find, it's also a piece of cake to write editor tools to jump between tests and their packages. Plus, for people new to the project, it's easy to learn. Whenever I work on a large project, I usually find the test suite to have some cryptic organization which only the maintainers know and I have to grep or ack through the tests and guess. With this system, it's quick and easy to learn.
Freelance Perl/Testing/Agile consultant and trainer. See http://www.allaroundtheworld.fr/ for our services. If you have a problem with Perl, we will solve it for you.
And don't forget to buy my book! http://www.amazon.com/Beginning-Perl-Curtis-Poe/dp/1118013840/
I like that idea a lot for unit tests, but I think it doesn't address testing coordinated behaviors that span multiple units. Still, moving away from numbered flat tests to meaningful names in a hierarchy makes lots of sense. There's no reason not to name behavior tests in a similar way.
All your files, not just the test ones, should have meaningful names. And sometimes, it makes sense to do the simple tests before the complex ones. And without leading numbers, the tests would be done alphabetically, which would mean you have to come up with names for the simple tests that are alphabetically before the complex ones. Which means you have to throw your meaningful names out the window. The best solution, so far, is to have leading numbers and meaningful names.
@dagolden: actually, it works fine for both unit and integration tests. You can even name them -unit.t and -integration.t if you want. I also worried that tests which span multiple units would not map well to this, but one of my colleagues pointed out that even for integration tests, there's a package or function which is the "end point", if you will, of the behavior under test. It has worked out very, very well for us and makes it trivial to understand the test suite. Otherwise, all naming conventions are ad hoc and prone to confusion.
@shawnhcorey: no, the numbers are almost useless and are an artifact of Test::Manifest and similar technologies not being used enough. See the post by Schwern that I linked to. As for "meaningful names", unless they map to packages (or maybe URLs for Web apps), they are almost guaranteed to be ad hoc names which lead to difficult to maintain test suites. They are a very, very bad idea.
This is why I like Test::Class (and recently, Test::Sweet). Class-based tests make it very easy to organize tests. I know that Test::Foo::Bar exercises the methods in Foo::Bar. And T::F::B can inherit tests from its parent, which parallels the parent of F::B.
With Test::Sweet, you can even compose test classes with both inheritance and roles.