Like most startups using perl, blekko began in 2007 using the perl that shipped with the Linux distro that we were using -- 5.8.8. Over the years, as our search engine and homegrown NoSQL database crawled towards a million lines of code, this ageing perl version frequently made us sad. We finally bit the bullet earlier this year, and here are the top 7 things we learned:

1. Upgrading is a good opportunity to review your test coverage!
2. Watch out for dual-life modules. We had upgraded quite a few, but to versions older than the ones shipped with 5.16. After running into a few unnecessary bugs, we removed all of downgraded modules.
3. Expect bugs when you update lots of CPAN distros - we use 400 of them. For example, BSD::Resource::getrlimit stopped returning a hashref in scalar context between 1.28 and 1.2904. Who knew?
4. Rerun all of your XS memory leak tests. (I'm hoping that Devel::SizeMe is really going to help us find the rest of the bugs in our XS memory management.)
5. Read the release notes. We found the comment about moving to SvUPGRADE from sv_upgrade to be very useful; it fixed a few long-standing bugs in our codebase.
6. All your hacks will haunt you. In 5.16, failed "requires" stay in %INC, which caused indigestion to our Map/Reduce system, which pickles up code to send it to run where the data is.
7. Don't forget to send out advice to your colleagues. We forgot to tell people not to use smart match or given/when... fortunately the cleanup wasn't too bad.

Right after 5.18 came out, we spent a few days working on it, and managed to come up with our first bug ever against a very recent perl: Bug #118159, where the value of truth for SVs with wildly different string and integer values behaved different from before. It was a recent accidental change, and was reverted. It was very exciting to contribute (even in a small way) to perl development at the tip!

(This material was part of my talk at YAPC::NA 2013. Oh, and a month later, we updated everything to perl 5.18 - it's good to be at the tip!)