Test-Simple, Test-More, and Test-Builder. What is breaking? Should I be scared?
Whats breaking? Much less than you might expect, but more than I had hoped.
Should I be scared? If you do not test your things now, and file reports for me to fix bugs, then yes, you should be VERY scared.
That about sums it up! (Just kidding.) But in all seriousness, PLEASE TEST YOUR CODE!. Please install the trial releases of Test-Simple, try building your dependency chains against it! Please also try your non-cpan code against it. There has been plenty of warning, and you still have almost 3 months to file bug reports.
Code that does any of the following is no longer supported:
- Replacing the $Test::Builder::Test singleton
- Monkey-patching Test::Builder (Note: Exceptions for some commonly monkey-patched subs)
- Directly modifying values in the Test::Builder instance hash
Early last year I adopted Test-Simple from Schwern. This includes Test::Simple, Test::More, and Test::Builder. Essentially this is the foundation that nearly all testing in perl and cpan is built upon. Initially my plan was to simply run maintenance and not shake things up. My plan failed.
Shortly after adopting the module I was asked by David Golden to apply a seemingly innocent patch. This patch changed the indentation of a comment intended for humans reading test results. Sounds simple enough right? WRONG. This single change caused a cascade of failures that ultimately would prevent most Moose extensions from installing. The root cause of this was the horrible state of testing modules intended to make testing easier.
This issue caused me to engage in a deep study of Test::Builder and tools that used it. Test::Builder was of course an amazing bit of engineering, it is no wonder the entire ecosystem is built on it. Schwern and the other contributors deserve a giant ++. That said, I reached the same conclusion most of them had a year or 2 earlier... Due to cruft and some unfortunate design decisions, most of the internals would need some form of rewrite to really solve the big problems.
My decision to undergo a major change was the second such decision. The first one was the Test::Builder/More 2.0 project. The 2.0 project was a complete rewrite from scratch, with the intentions of then updating the old tools later to use the new stuff. This project saw a lot of good development, but ultimately petered out. The way it was explained to me is this, the 2.0 project blocked Test::Builder development. Everyone devoted attention to the new stuff, which ultimately didn't happen, and nobody gave any love to what we already had.
I decided to take a very different approach. I was not going to write things completely new, or from scratch. I was going to refactor things, preserving compatibility wherever possible. Some design decisions needed to be altered, and that has resulted in some incompatibility, but these are minimal, and have effected relatively few modules, many of which have been fixed already.
Cool story bro, but what else do I REALLY need to know?
- The Test::Builder singleton is no longer the central hub
- Subtests were the root of all evil, they have been re-implemented
- The ability to test your testing tools was the biggest focus
- Most things that previously required monkey-patching or other hacks can be done via an API now
Here are some docs for anyone who actually wants to see what is new: