I need to do two things:
- Think of a better name than Test::Class::Profile
- Write it
The problem: running several thousand tests in one Test::Class process can swallow up a lot of information. I want to instrument Test::Class to optionally capture that information to assist debugging/management of your test suite. I need to write code which will:
- Tell you what classes ran and in which order
- How long did each class take to run
- How long does each test control method take to run
- Overhead on test methods from test control methods
The last one is rather interesting. Consider a test class with a setup method which takes 8 seconds to run. If it has 10 test methods, than that's an extra 80 seconds of overhead you've added. If you can get the setup method to run in 4 seconds, you can save 40 seconds on the test class. Do this with just a few test classes and you're talking about serious performance savings.
What other information would you want in something like this? How should the information be recorded/presented?