Chad 'Exodist' Granum

The new Test-More stuff will NOT be part of 5.22 as originally planned. Instead the latest stable with ship with 5.22. The reason for this is a perl-qa consensus about a design decision that should be changed.

Back when performance was a problem I tried out using array based objects instead of hash base ones. In profiling, the object system was one of the big time eaters. While I changed the base to arrays I also changed how the accessors were built, and how some other parts of the objects worked. In the end my profiling showed huge improvements. Unfortunately I failed to measure the array base and other optimizations independently. I also failed to record my data, making it impossible for me to provide a sound argument for the array based design.

I got called out on this, and was urged to consider switching back to hashes. I decided to go ahead and get some hard numbers with which to make a decision. Ultimately it turns out that the gains of using arrays over hashes are very minuscule. When I was doing my earlier profiling I failed to notice that my other optimizations (use of constants, better/cheaper accessors, etc) were where the real performance enhancements were being seen.

Anyway, here are the hard numbers:
http://test-more.github.io/test-more/

Based on these numbers there does not appear to be a compelling reason to continue to use array based objects. However there are plenty of reasons to prefer hash base objects. Hash based objects can be dumped, are easy to read, and are what people expect. It also makes extending the objects easier.

As you can see I have already done the work in branches to switch to hashes, but this is just a direct switch. The direct switch fails to update/alter api's to work better with hashes, as such he branches currently wrap/hack the architecture change instead of embracing it. I would like to take the time to actually reflect the new object base in the api's to provide a better user and developer experience. I do not think it would be wise to rush that work just to have it in 5.22, instead I will take the time to do it correctly, and make sure it is up to snuff.

I would encourage anyone who has interest to reproduce my results. All the branches and profiling scripts are present in the github repo.