A call to action for CPAN authors
CPAN authors should look at the smoke tests for their modules to ensure that they're passing on Perl 5.18. The hash randomization change (and a few others) has bitten many a module that may currently be relying on undefined hash behavior. If you haven't checked your modules recently, you may be in for a surprise.
The problem is significant, especially when compounded by the fact that a single misbehaving module can block many more modules that may depend on it.
As an example (not to finger-point, but just to illustrate): Crypt::DES, Crypt::IDEA, Crypt::Blowfish, and Crypt::Twofish had a single line of XS code that was incompatible with Perl 5.18 (which happened to not be related to hash randomization). As a result, all modules that depended on any of these modules also would fail to install on Perl 5.18. This includes Authen::Passphrase, and Crypt::OpenPGP, as well as dozens of others.
For awhile during Perl 5.17 development Perl 5 Porters were keeping a list of modules they were aware of that had bugs made evident by the hash randomization change. But given that we've been hearing warnings about relying on undefined behavior for years, and in particular warnings about hash randomization for many months, at some point it stops being Perl 5 Porters responsibility to keep track of modules that have bugs.
In many cases, these bugs are in modules' test suites, and not the modules' primary code. But whatever the case, we need to be diligent in ferreting out such problems and resolving them. Having a good portion of CPAN break because of a number of modules relying on undefined behavior is one of those "egg on our face" moments that we need to wipe off and put behind us.
If you know one of your modules has begun failing on Perl 5.18, I encourage you to fix it. If you know of an unmaintained module that has begun failing, follow up with the author, or seek to take over maintenance. Report problems to the RT system. Contribute patches. Let's work together to get this problem resolved as quickly as possible.
daoswald [a@t] gmail [d0_t] com
oh, we made a breaking change, and its youuuur fault your code is broken!
I think I understand that position. But what does it profit us?
We're here now, it's not going away. However, we've all benefited from the labor and intellectual capital represented by CPAN. And I'm sure we have people in the community willing to help.
I'm not sure I would have been so sarcastic, however the posting had me asking a similar question.
I hope a change that breaks so many CPAN modules was vetted by the community before it was promulgated (my guess is it was). I am not as plugged into such cutting edge versions like 5.18 of Perl as I tend to use the most stable version include in standard distros.
Hopefully, the first comment will tease out some discussion of the broader question regarding how such changes are decided upon and what benefit they provide the community when clearly there will be many people unable to use the version if dependencies are busted.
OTOH, you have to break some eggs to make an omelette.
;-)
I believe the 'breaking change' was for security reasons; to prevent an horrific DoS situation that had been discovered recently.
As I understand it, getting hash data back in a predictable order is a bug that people have learned to rely on - but still a bug.
Hash-based tests really ought to be using "cmp_deeply($foo, bag(...))" or "sort keys %hash" idioms.
This change revealed bugs in test-suites; yes many of us fell into the same trap, but I'd argue that is *is* the module author's responsibility to fix the bug in their code that's been masked by the bug in perl's hash ordering.
Further reading:
- http://perldoc.perl.org/perlsec.html#Algorithmic-Complexity-Attacks
- http://lwn.net/Articles/474912/
"Entries are returned in an apparently random order. The actual random order is subject to change in future versions of perl, but it is guaranteed to be in the same order as either the keys or values function would produce on the same (unmodified) hash"
Perl 5.8.9 documentation, Dec 2008.
(http://perldoc.perl.org/5.8.9/functions/each.html)
@James: We only broke modules that were already subtly broken. In certain circumstances, Perl would actually change the order of the hash contents (rehashing due to collisions or hash growth) and two hashes with the same content could actually have differing hash orders. It's just that it was rare and lurking to make your mission critical software fail in mysterious ways in the most unfortunate moment. You're welcome.
On top of that, we had a one-liner that could make any unpatched Perl consume a virtually infinite amount of RAM. This was through an HTTP request if the process on the other end put query parameters in a hash. Ouch. Again, you're welcome.
I'm not sure what the confusion here is...
AFAIK this change has been spoken about for quite a while, at least a year that I can recall. Beta and RC releases have been out with this change, and if you are a CPAN author you've likely already been getting reports about the broken test cases for months. So I can't really think its fair to jump in at such a late moment and complain about the 'decision making process'.
A slightly backwards incompatible change was made that made already broken code slightly more obviously broken in order to fix a monumental security issue. This is pretty big deal for the Perl community, since in general we have a very hard time making choices like that. If anything I think the pace of change is dooming Perl...