A few lessons from updating to Perl 5.16
Like most startups using perl, blekko began in 2007 using the perl that shipped with the Linux distro that we were using -- 5.8.8. Over the years, as our search engine and homegrown NoSQL database crawled towards a million lines of code, this ageing perl version frequently made us sad. We finally bit the bullet earlier this year, and here are the top 7 things we learned:
- Upgrading is a good opportunity to review your test coverage!
- Watch out for dual-life modules. We had upgraded quite a few, but to versions older than the ones shipped with 5.16. After running into a few unnecessary bugs, we removed all of downgraded modules.
- Expect bugs when you update lots of CPAN distros - we use 400 of them. For example, BSD::Resource::getrlimit stopped returning a hashref in scalar context between 1.28 and 1.2904. Who knew?
- Rerun all of your XS memory leak tests. (I'm hoping that Devel::SizeMe is really going to help us find the rest of the bugs in our XS memory management.)
- Read the release notes. We found the comment about moving to SvUPGRADE from sv_upgrade to be very useful; it fixed a few long-standing bugs in our codebase.
- All your hacks will haunt you. In 5.16, failed "requires" stay in %INC, which caused indigestion to our Map/Reduce system, which pickles up code to send it to run where the data is.
- Don't forget to send out advice to your colleagues. We forgot to tell people not to use smart match or given/when... fortunately the cleanup wasn't too bad.
Right after 5.18 came out, we spent a few days working on it, and managed to come up with our first bug ever against a very recent perl: Bug #118159, where the value of truth for SVs with wildly different string and integer values behaved different from before. It was a recent accidental change, and was reverted. It was very exciting to contribute (even in a small way) to perl development at the tip!
(This material was part of my talk at YAPC::NA 2013. Oh, and a month later, we updated everything to perl 5.18 - it's good to be at the tip!)