The PTS proved to be a very valuable event, as it always has in the past. I am glad it is able to occur again in the wake of covid. One of the first things the group discussed was the lack of motivation in recent years when we could not meet, and how simply coming together again solved that problem for most of us.
This year my employer Grant Street Group paid for me to go to the PTS!
The first day largely became about important discussions for the toolchain group. Some included a majority of attendees, others were composed of smaller subsets depending on the topic.
Legacy Target Burden
The first discussion was over the legacy burden that toolchain developers must bare. Since the Lancaster accords we have agreed to target perl 5.8.1.These releases are well over 10 years old, and are used by less and less people as time goes on. In addition, perl continues to get better and introduce useful constructs that we have been unable to use.
In previous years the idea of raising the target perl version was very contentious. One of the main reasons for legacy support is to avoid driving away perl users who do not upgrade perl, but find stuff breaking if they upgrade cpan modules. One of the stronger points of perl is that it does not break things on a whim, and code written years, or even decades ago often works with little or no modification even on the newest perl versions.
Much to everyone's surprise, there was very little disagreement that the target could and even should be raised. Further we all agreed there should be a policy that regularly raises the target so that this discussion does not have to occur again.
The final decision was that we will set the target to be whatever version of perl was released 12 years ago this year, 11 years ago next year, and 10 years ago moving forward. This decision was because we all agreed 10 years was the ideal target, but we wanted to make a one-time exception for a version of Redhat/Centos that is still in common use, and supported, but will end its support window in 2 years.
EU Regulations for software
This discussion was about 2 regulations the EU is passing (or may have passed at this point) that effect companies that write software. You should read someone else's blog if you want a detailed explanation. My summary is this: It makes writing software risky in the EU. It is not supposed to effect Open Source Developers, but everyone is suspicious of that. It does not effect non-EU countries like the USA, UK, Canada etc, but there is some concern of indirect effects.
Yath Discussions
Several members of the PTS came to me to discuss yath, and how they use it either in open source projects, or at their workplaces. I was surprised to find out how wide-spread yath has become. I helped a few people get some bugs fixed and minor features added. I also talked through some upcoming plans I have for yath to improve its efficiency. As it stands yath gets inefficient when you use it on a very powerful machine with many cpus. The IPC system, scheduler, and most notably the collector become significant bottlenecks.
My Plan is to try to release Yath 2.0 in the not too distant future, with a revamped IPC system, and a more streamlined pipeline system without the current bottlenecks. I managed to do some work on this while at the PTS, but not anything that is ready for release yet.
Additional Notes
My second day at the PTS was largely spent working on Test-Simple and Test2-Suite bugs.
I also had discussions, and worked with Paul Evans (LeoNerd) on adding some refcount tools to Test2::Suite.
I also participated in a discussion around cpan overlays, alternate indexes, etc. that would make it possible for people who are on perls older than what the toolchain supports to continue to function. Ultimately the goal is to have a way to tell cpan, cpanm, and other cpan clients to use an alternate index that does not include any cpan package that requires a higher perl than the one that is installed. My chief contributions to this discussion are as follows:
Most of day 3 was spent working on bugfixes, the changes for refcount, and yath. There were also further discussions about cpan indexing.
Travel Issues
Day 4 was a little crazy because several members of the PTS had their flights canceled due to the French protests. Everyone was scrambling to make new travel plans, or have backup plans ready. Several people ended up adding extra hotel days. I spent a couple hours checking plain, train, and bus options to get to Amsterdam, or alternatively Geneva in case my flight from Lyon to Amsterdam was canceled. All indicators said my flight was not canceled, but I was not able to check-in, the system refused and said I needed to do it at the airport.
Ultimately I decided to check out of my room early that evening (too late for a refund) and get a room at the hotel at the airport. That allowed me to go check-in in person before going to bed. I had a bit of a panic as the first kiosk I tried took all my info, then said "please wait" until it timed out. The second kiosk made me start over and asked me to pick my next hop (Amsterdam) but unlike the first kiost it refused to let me pick it! I grudgingly tried a third kiosk, and it worked flawlessly and let me check in and printed my
boarding pass.
Between all that Chaos I was able to have a few more productive discussions about various parts of the perl toolchain. I was also able to get some more work done on yath.
Parting Thoughts
It was really good to be back at the PTS. We were able to accomplish a lot, and we all got a serious moral/motivation boost. We managed to discuss some very important topic, and even better reach agreements on how to handle them moving forward. This was an extremely valuable event for perl, and for any person or organization that depends on perl.
]]>I maintain several perl modules, if you depend on any (Such As Test-Simple, Test2, or yath) here is a way to show your support :-)
https://www.patreon.com/exodist
]]>Because this is such an important module, and this release is such a big change, I have also prepared extra tarballs for Test2 and Test-Simple that completely revert the merge and bump the version by one. If a showstopper or emergency occurs I can upload them in an instant to put things back to mostly the way they were.
I am not anticipating any big issues with this. There has been a lot of testing with a huge focus on backwards compatibility. That said, nothing is ever completely bug free, specially when it is as big as this. If you find any issues please report them!
Along with this distribution I have released the following distributions as stable (up until now they have been listed as experimental)
]]>Since that release a number of minor bugs have been fixed. Most of these are typo fixes and documentation changes. There were also some mistakes in our x-breaks meta-data that needed to be corrected. Finally there was one bug fixed that was not as minor, a 64-byte shared memory leak when using threads.
I have decided that these fixes, in particular the last one, are important enough to put out a new dev release, and bump out the stable release date. 1.302014_010 will hit cpan today. Assuming no other substantial bugs are found the new release date will be Tuesday May 10'th. Obviously being correct is more important than releasing by a specific date, so if significant items are found the release could be delayed again.
If you find any bugs or issues please report them right away at https://github.com/Test-More/test-more/issues.
]]>Test2 and Test::Builder remain separate. Tools pick one a build with it. To use both together you load a shim such as Test2::Legacy, which replaces Test::Builder with the alternative guts.
This was the alternative being discussed in the Test2 meeting on day 1 of the 2016 hackathon. We decided it was not the right plan. My main reason for opposing this plan is that nothing stops people from updating old modules to use Test2, and also auto-loading the shim to insure they do not break things that depend on them.
In this scenario you can never be sure that updating your test modules will not make the choice for you. If you use Test::Moose::More for instance it is currently on Test::Builder. Lets say the author updates Test::Moose::More to use Test2. If he does so he will break a lot of tests that also use other Test::Builder modules. So in order to avoid the problem he loads Test2::Legacy automatically. Now when you update Test::Moose::More, suddenly all your .t files load Test2::Legacy even though you did not directly ask for it.
In this scenario you have no idea when or if the switch will happen unless you watch all your test deps like a hawk every time you update from cpan. You have no idea which pebble will fall causing many of your test files to avalanche into Test2.
The proposal we went with instead was to make new Test::Builder use Test2. This means that there is only 1 point for the switch to happen. If you update Test-Simple then you get new guts. If you do not update you get old guts. If you pin your Test::Builder version than any downstream deps that require Test2 will fail to install and you do not need to worry about accidental updates.
The proposal we adopted means there is a single point of concern, which Test-Simple version you have installed. With the alternate that I myself mentioned back in 2015 in Berlin, and was still on the table table last week, you have no idea when or if your deps will change and suddenly switch you unwillingly.
This goes further. If we update Test::Builder to use Test2 (the current plan) than there is one thing for Test tool authors to target. If we keep them separate they need to insure their modules work with both the old and the compatibility shim, or they can just choose to break without the shim. This would lead to any number of possible combinations of test modules that can just explode one day due to simple changes.
Making Test::Builder use Test2 means we have 1 breakage event. Using an optional shim means we have any number of breakage events in the future as people maintain the test tools. The only way to prevent that would be to not have a shim, and force people to maintain both namespaces. But if I don't release the shim I am certain someone else eventually will, and then we have the same problem, possibly with even more things breaking if the shim is not as good.
]]>For the impatient, here is a summary:
The QAH discussion focused around a single question: "What is the best path forward for Test::Builder when we consider both end-users and test tool authors?"
Arguments for updating Test::Builder to use Test2:
Arguments for keeping Test2 and Test::Builder separate:
When the room came to a vote the result was overwhelmingly in favor of updating Test::Builder.
The discussion also resulted in a few action items that prevent a stable release during the QAH:
These are important action items that help to alleviate the concerns of those who were against updating Test::Builder. All of these items are now done, and a dev release has been uploaded to cpan. Assuming the dev release does not find any show-stopping bugs it will be released as stable no sooner than Friday May 6th which is the planned release date.
Thank you to DreamHost for sending me to the QAH! ]]>
Here are the big things people wanted:
We decided that the best way forward was to forget about compatibility with Test-Stream, which is still new enough that it is not seeing much use, and make a new name with everyone’s changes. That’s Test2.
After the split there are 3 distributions:
Test2::Suite and Test2::Workflow are optional systems. I think you’ll find them very useful, but they’re not the key deliverable. That’s Test2, which is meant to replace Test::Builder’s guts. Test::Builder will be a backward compatibility layer on top of Test2.
As of the latest releases of Test2 and Test2-Suite I am considering the work done to my specifications and intentions. Now is the time to review these and make commentary. Once this goes stable it will become much harder, if not impossible, to make sweeping changes. If there is anything in these distributions (Test2 in particular) that makes you uncomfortable, you need to speak now. I am not yet ready to say Test2-Workflow is complete.
The work to make Test::Builder use Test2 under the hood is also complete.
On CPAN: https://metacpan.org/pod/release/EXODIST/Test-Simple-1.302013_011/lib/Test/Builder.pm
On Github: https://github.com/Test-More/test-more/tree/Test2/master
Blead Patch Ticket: https://rt.perl.org/Public/Bug/Display.html?id=127000#txn-1385453
This work has been through several verification procedures:
Note: The preferred form of feedback is tickets on the relevant github projects.
]]>With Exporter, and most exporter tools we have failed to separate concerns.
Exporting fundamentally involves 2 parties: exporter and importer. Historically
however we have only ever concerned ourselves with the exporter. The "standard"
way of dealing with this in perl has been to have a module that provides
exports use it's import()
method to inject symbols into the importer's
namespace.
What if we did this with other similar concepts? What if instead of:
use base 'foo';
we had:
package My::Base;
# Gives us an import() method that sets @{caller::ISA} = __PACKAGE__
use base;
...
package My::Subclass;
use Base::Class; #Automatically sets @ISA
Most of you are probably cringing right now. I know in the past I have actually used this pattern and gotten slapped on the wrist for it (rightly so). But this is essentially what we are doing with our current exporters.
This has lead to many tools to try and work around the madness. We have several Exporter alternatives, some provide an alternate way of specifying exports, some provide better features for the importer, and some do both. What if you like Exporter::Declare's interface for creating an exporter, but much prefer Sub::Exporter's interface on the importing side? Tough luck, pick one. We don't have this problem when picking between 'parent' and 'base' for subclassing.
We CAN fix this problem. We need to separate our concerns.
I have uploaded the Importer module on cpan. This module acts on behalf of the consumer of exports. Instead of
use Carp qw/croak/;
do
use Importer 'Carp' => qw/croak/;
What this does is turn Exporter into a specification for declaring that a
module has exports. Subclassing Exporter, or importing its import()
method
are no longer required though. Importer is 100% compatible with all current
Exporter features and variables. Because of this you can now have a module that
specifies exports, but does not have an import()
method, or has an
import()
method that does something else completely.
Importer has 0 non-core dependencies. Even core dependencies are used sparingly. The only use/require statements it contains are for strict, warnings or to load the module that has the exports (on demand).
The Importer module also has a hook that should allow most other exporter modules on cpan to make themselves compatible with it. I will update Exporter::Declare in the near future. Other exporters will NOT have to use the package variables Exporter expects to work with Importer. You can even directly use the hook in your module to specify your exports (it is just a sub you define in the exporting package)
Note: This part is not essential. I will be perfectly content just having Importer on cpan.
I have a perl-blead patch in the works that puts Importer into blead, further it updates Exporter in the following ways:
This patch currently causes 1 test failure, and based on a conversation with #p5p it is not likely a real problem, just some large strings that need to be updated (fragile test). That said nobody has touched the test in years and we are not completely sure yet.
I have also tested this change against my canary list of cpan modules, and the current iteration works fine for all of them.
Having Exporter defer to Importer for anything more complex than a 1-to-1 sub import has the benefit that the modules will not grow out of sync. This is helpful, but not required, for maintaining compatibility. It also makes it a lot less likely anyone will ever have to touch Exporter again.
I would also be perfectly happy combining Exporter and Importer into a single dist, and I have no strong opinions for if it would be p5p upstream, or cpan upstream. This depends entirely on what p5p wants to do, they could also just outright reject the proposal.
While working on my Test::Stream/Test2 work I found myself needing the ability to rename subs I imported. This was frustrating as Exporter does not support this feature. In addition the exporters that do are non-core, and not likely to ever be core (Test::Builder can't use non-core modules). I initially solved this by writing my own exporter as part of Test::Stream, but this generated significant negative feedback. Ultimately I was able to get around the problem completely, but I still wanted to solve it as a separate project.
A while back I submitted a proposal and patch to p5p for an enhancement to Exporter. All in all the reception to the idea of the new features was good. There were a lot of opinions on how it should be done, or what side effects there may be, but nobody seemed to outright reject the idea of the features. The things that were brought up really did stick we me though, in particular:
Burden of dependency
To use the new features the consumer is obligated to list the newer Exporter version as a dep, this seems backwards.
import() modifying consumers namespace
Having an exporter work by modifying the consumers namespace seems backwards,
we don't have base classes modify a consumer's @INC
on import, we have
parent and base.
The thread fizzled out partly due to these concerns, partly because the patch did not go far enough in adding the desired features (some people on p5p wanted it to do more), and also a lack of time/energy on my part due to Test2/Test-Stream work. Lately I had some spare time, and I decided to come back to this problem from this new angle.
]]>I will try to be as clear and concise as possible:
There has been a lot of FUD about this lately, I don't think any of it was malicious, but rather due to a misunderstanding. Test::Stream is a new namespace, and includes a new toolset modeled after Test::More. This new toolset has a new implementation of is(), and for a while did not have a cmp_ok(). These differences are opt-in, you have to use Test::Stream instead of Test::More to be effected. There are no plans to ever port these differences to Test::More, that would break things, and I am not willing to do that.
So what is changing?
My goal is to change Test::Builder so that it defers to the Test::Stream internals. The idea is to do this in ways that break as little as possible. The current Test::Builder dev release on cpan does not break anything that properly uses the public api. It also does not break most things that deviate from proper use of the public API. Last I checked there are only 8 distributions on all of cpan that are broken by these changes, and none of them have significant usage. Most, if not all of these breakages have perl-qa's agreement that they broke for being too crazy to support.
So to be clear: Backwards compatibility is one of the highest priorities. If you use Test::More, or other tools built on Test::Builder, they should continue to work.
If you are interested in seeing what is broken I have a list that I frequently use in downstream testing. This list currently has 9 items, but one of them was recently fixed, I have not removed the fixed item yet because I want to look at it and see how it was fixed. The list can be found here: https://github.com/Test-More/test-more/blob/stream/master/xt/downstream_dists.list.known_broken
]]>After this was written I received feedback from several respected members of the community alerting me to the problems that could be caused by is() *guessing* if it should be comparing numbers or strings. After hearing this feedback I agreed that the behavior constituted a bug, and one serious enough to alter the behavior post release. Test::Stream was marked *stable* recently enough that the change should not impact very many people, if in fact any.
The latest version of Test::Stream on CPAN no longer guesses if it is given a number vs a string.
In the very near future cmp_ok, and a 'Classic' bundle which provides just the functionality of Test::More, including classic 'is' and 'is_deeply' will be released, and will be the recommended bundle for people moving from Test::More. Branches and pull requests for both of these have been written, I am giving them some time for review before proceeding with their release.
'use Test::Stream;' without any import arguments throws an exception. There are no defaults. You, the user of the tool, must make a choice. This is achieved through the loader/plugin/bundle system, partly influenced by Dist::Zilla.
The recommended bundle has these ideas in mind:
Very few people will complain about the first 2 ideas. The third idea is likely sending chills down some spines. The fourth idea will be explained later. One of the first things I did was look at what I saw as common pitfalls. These are things that often get questions on irc, reported as bugs, etc. Here are the ones that have caused the most commentary:
I was surprised at the backlash against not having 'cmp_ok'. The arguments I have received here have caused me to reconsider, I will likely be adding cmp_ok to Test::Stream. If someone beats me to it, and their implementation is sound, I will happily incorporate it.
I knew that changing is() would ruffle some feathers. However I think that new people, put two things into 'is()', and expect it to say if they are the same. Thinking about the fact that they are comparing numbers, strings, hashes, etc. is an unexpected burden on new test writers. Once they are experienced they will learn WHY thinking about it is important, at which point they can use the Compare plugins advanced usage to be clear what they want.
Ultimately this line of thought lead me to create the Test::Stream::Plugin::Compare plugin. Note, the implementation of 'is()' is it's own plugin, it is not part of the 'Core' plugin. This is important, I knew that some people would disagree with my implementation (my third to be precise). I expect alternatives to appear, and I hope they are better than mine. I felt that putting it in its own plugin made it seem an open invitation to alternative implementations, as opposed to calling it 'Core'.
I made the Compare plugin work in what I felt was the most useful way for an average person. This library compares whatever you give it, if you give it string it will compare them as string, numbers are checked as numbers, if you give it references it will compare the references, structures like hashes are compared deeply.
There are a couple things in people would take exception with here:
Following the ideas outlined further above I put more weight on the second issue here. You can never be sure, but you can make very accurate guesses. looks_like_number from Scalar::Util is pretty good for this. I further added 2 more features to help:
Here is an example on using this implementation of 'is' as 'is_deeply', and loading your own 'is':
use Test::Stream::Plugin::Core;
use Test::Stream::Plugin::Compare is => { -as => 'is_deeply' };
use Test::Stream::Plugin::MyIsPlugin;
Here is the same thing, but using the loader:
use Test::Stream 'Core', Compare => [is => { -as => 'is_deeply' }], 'MyIsPlugin';
Of course that is still a lot to type at the start of all your tests, so you would create the MyWay bundle that you use in all your tests:
use Test::Stream -MyWay
Back to Idea 4: "It should have what is needed to provide examples that teach people the right way to do something people usually do wrong when they first try."
I included 'dies { ... }' and a similar tool for warnings in the recommended bundle. This was done so that I could include examples of their use. History shows that people have hit the Test::Exception problem multiple times, with things like Sub::Uplevel produced to solve them. Then something glorious happened, Test::Fatal, which showed us a better way. By including these I can provide examples early on to new people so they do not fall into the same trap.
That was my justification for including them, though I have since relented. I expect that the next recommended plugin (-V2 if/when I add it) will likely not include them. Instead it will recommend Test::Fatal directly.
Notes:
Some may wonder why I am including my Warnings and Exception plugins in the main dist at all. There are probably other plugins that you might ask about. Ultimately my decision for which plugins were included in the main dist boiled down to which ones were essential to testing Test::Stream itself. If a plugin is in the main dist it is because I felt it was essential to testing Test::Stream. I need to test my exceptions, I need to test my warnings. Test::Fatal is not core, and I don't want to duplicate exception checking code over and over again.
This blog post reflects the entirely new module 'Test::Stream'. This document does nto address Test::More in any way. The Test::More API will not change at all. Test::More will not be altered to use Test::Stream plugins. The planned changes for Test-Simple involve changing Test::Builder so that it sends events to the Test::Stream infrastructure instead of directly writing TAP. The actual guts of the tools will change very little.
Why not include an 'is()' implementation like Test::More's in the 'Core' plugin? Because I felt its stringification behavior, while expected by people who have used it a long time, is a tripping hazard for new testers. I would rather leave Core unopinionated on 'is', and let that battle be fought in the plugins and bundles people create.
I fully expect some people to disagree about decisions made for the recommended bundle. This is fine, it is probably not intended for you.
]]>What does this mean for Test::More, Test::Builder or my existing test tools?
At the moment it means very little. Test::Builder is still around, and not going anywhere any time soon. At the moment Test::Builder based tools and Test::Stream based tools will not work together in a single test script, though they can both be used in different files in a single test suite.
Will Test::Stream and Test::Builder tools work together?
Not with the current stable Test::Builder release. However the latest developer release changes that. With the developer release most Test::Builder and Test::Stream tools will play nicely together. The exception is with tools that rely on replacing or modifying the Test::Builder singleton itself. Tools that modify the Test::Builder singleton will still work as expected for most situations, but will likely have no effect (desired or undesired) on tools that do not also use Test::Builder.
What is the future for Test::Builder?
The current dev releases of Test::Builder change it so that under the hood it uses Test::Stream. These changes are fairly significant, and could be seen as scary. These changes effectively turn Test::Builder into a compatibility wrapper around Test::Stream. I plan to release these changes once the perl-qa group, Ricardo Signes, and I all agree it is ready and stable. This process is called the "punchlist" and was devised at the 2015 perl-qa hackathon in Berlin. Several things have changed, so the punchlist is no longer completely accurate, but is good enough to explain what has to happen before these changes go stable.
How can I help?
Start writing new testing tools that use Test::Stream. You could also update existing test tools to work as plugins for Test::Stream. At the moment a full conversion of an old tool is not recommended, since Test::Stream and Test::Builder do not yet work together you can cause a lot of headaches and breakages by simply converting a tool. Instead it would be better to abstract the tools logic so that the main module continues to use Test::Builder, and the plugin form (Test::Stream::Plugin::YourModule) uses Test::Stream.
You can also help by testing the Test::Builder dev release. 'cpanm --dev Test::More' will install the dev release, and the latest version of Test::Stream along with it.
Where can I find out more about Test::Stream?
Why should I use Test::Stream?
Test::Builder and Test::More have been around for a long time, and have proven their worth. On the other hand it has been years since they saw any significant development. In addition there is a multitude of tools that have to hack Test::Builder to make it do what they want.
Test::Stream addresses most, if not all, the limitations and complaints about Test::Builder. Test::Stream has more capabilities and hooks for people writing tools. In addition Test::Stream provides an improved API for people writing tests, one that still looks and feels like Test::More, but makes changes that cannot be made in Test::More itself for compatibility reasons.
Before diving into details, I want to thank my employer, DreamHost for sponsoring my attendance at the hackathon, they paid for my travel and hotel. DreamHost also went a step further by providing the perl-qa group with a dreamcompute account we were able to use to do smoke testing among other things.
So, what did I accomplish at the hackathon? The simple answer is Test-Simple.
On the first day of the hackathon we had a group discussion related to Test-Simple (Test::Builder, Test::More, etc). This discussion was made up of 3 parts:
1) What from Test-Simple could be improved? If Test-Simple is going to be revamped, what things should change? Who would benefit from these changes? Are the benefits enough to outweigh the risk in changing Test-Simple.
2) Should Test-Simple be modified in-place, or should we start fresh in a new namespace?
3) Does the implementation I have been working on sufficiently address the concerns and issues discussed in #1? Is the implementation sane at a high level? Does anyone have any glaring objections to the Test-Stream design?
We created a nice list of things that needed improvement, and new features we wanted to have, we determined that there were many groups of people that could benefit from change. We discussed the pros and cons of forking into a new namespace, and ultimately decided that changes could go into the existing namespaces. Finally it was determined that at a high level my implementation sounded sane. In the end we produced a punch-list of things that should happen before my implementation could be released.
The punchlist can be found here. Several items have already been completed, but a handful are still outstanding. In addition the right is still reserved to block shipping if anything crazy is discovered during the punch-list process. Ultimately I plan to be patient, and not rush things, even once the punch-list is complete.
]]>This blog post is a cry out for review. Test-Builder/More/Simple are being seriously reworked. From my perspective as the author of these changes everything looks fine, but that is just one set of eyes.
How can you help?
The git history on master does not lend itself well to review. This is a harder problem to solve. I have been urged by a couple people to rewrite history to make things more clear. I will take the time to make a cleaner branch where history is easier to read. Assuming the new branch works out, and accurately represents history, I will rename master to something else (but always keep it so that people can see it)
Back when performance was a problem I tried out using array based objects instead of hash base ones. In profiling, the object system was one of the big time eaters. While I changed the base to arrays I also changed how the accessors were built, and how some other parts of the objects worked. In the end my profiling showed huge improvements. Unfortunately I failed to measure the array base and other optimizations independently. I also failed to record my data, making it impossible for me to provide a sound argument for the array based design.
I got called out on this, and was urged to consider switching back to hashes. I decided to go ahead and get some hard numbers with which to make a decision. Ultimately it turns out that the gains of using arrays over hashes are very minuscule. When I was doing my earlier profiling I failed to notice that my other optimizations (use of constants, better/cheaper accessors, etc) were where the real performance enhancements were being seen.
Anyway, here are the hard numbers:
http://test-more.github.io/test-more/
Based on these numbers there does not appear to be a compelling reason to continue to use array based objects. However there are plenty of reasons to prefer hash base objects. Hash based objects can be dumped, are easy to read, and are what people expect. It also makes extending the objects easier.
As you can see I have already done the work in branches to switch to hashes, but this is just a direct switch. The direct switch fails to update/alter api's to work better with hashes, as such he branches currently wrap/hack the architecture change instead of embracing it. I would like to take the time to actually reflect the new object base in the api's to provide a better user and developer experience. I do not think it would be wise to rush that work just to have it in 5.22, instead I will take the time to do it correctly, and make sure it is up to snuff.
I would encourage anyone who has interest to reproduce my results. All the branches and profiling scripts are present in the github repo.
]]>That about sums it up! (Just kidding.) But in all seriousness, PLEASE TEST YOUR CODE!. Please install the trial releases of Test-Simple, try building your dependency chains against it! Please also try your non-cpan code against it. There has been plenty of warning, and you still have almost 3 months to file bug reports.
What breaks?
Code that does any of the following is no longer supported:
There are a few monkey-patching exceptions, if your code wrapped 'ok', 'diag', 'done_testing' or a couple other common ones, some fancy hackery has been done to allow them to keep working.
Backstory:
Early last year I adopted Test-Simple from Schwern. This includes Test::Simple, Test::More, and Test::Builder. Essentially this is the foundation that nearly all testing in perl and cpan is built upon. Initially my plan was to simply run maintenance and not shake things up. My plan failed.
Shortly after adopting the module I was asked by David Golden to apply a seemingly innocent patch. This patch changed the indentation of a comment intended for humans reading test results. Sounds simple enough right? WRONG. This single change caused a cascade of failures that ultimately would prevent most Moose extensions from installing. The root cause of this was the horrible state of testing modules intended to make testing easier.
This issue caused me to engage in a deep study of Test::Builder and tools that used it. Test::Builder was of course an amazing bit of engineering, it is no wonder the entire ecosystem is built on it. Schwern and the other contributors deserve a giant ++. That said, I reached the same conclusion most of them had a year or 2 earlier... Due to cruft and some unfortunate design decisions, most of the internals would need some form of rewrite to really solve the big problems.
My decision to undergo a major change was the second such decision. The first one was the Test::Builder/More 2.0 project. The 2.0 project was a complete rewrite from scratch, with the intentions of then updating the old tools later to use the new stuff. This project saw a lot of good development, but ultimately petered out. The way it was explained to me is this, the 2.0 project blocked Test::Builder development. Everyone devoted attention to the new stuff, which ultimately didn't happen, and nobody gave any love to what we already had.
I decided to take a very different approach. I was not going to write things completely new, or from scratch. I was going to refactor things, preserving compatibility wherever possible. Some design decisions needed to be altered, and that has resulted in some incompatibility, but these are minimal, and have effected relatively few modules, many of which have been fixed already.
Cool story bro, but what else do I REALLY need to know?
Here are some docs for anyone who actually wants to see what is new:
]]>