A Porter's Tale

Well, I’m just about done with my porting project.  For $work, we’ve been working on porting a fairly large codebase (the official count is nearly 2 million LOCs, although I think that probably includes whitespace and comments) from Perl 5.8.9 to 5.14.2.  Actually, our goal is threefold:

  • Get our code using the newest Perl.  (Although, of course, that’s always a moving target: by this point 5.16 is out, so it’s not the newest any more.)
  • Get our code using the newest Moose.  (We’re currently on 1.21, and want to go to 2.x.)
  • Switch from using MooseX::Method::Signatures to using Method::Signatures::Modifiers.  In fact, this port was the primary impetus for me writing MSM in the first place (although, back then it was Perl 5.12 we were aiming at).  Now, in a bizarre sort of paradoxerie, the desire to get away from MXMS is the driving force behind completing the port.
I’ve been working on this port for several months now, although of course other projects have intruded here and there, and part of the time is me just waiting for our excellent QA department to find another problem.  And they’re still working on it, so technically we still might find something else, but the last few rounds of “problems” have just been merge conflicts and that sort of thing, so I’m feeling pretty confident that we’ve flushed out the majority of the issues now.  I thought it might be instructive to share the issues that I’ve seen during this process, in the hopes that it helps someone out who might be traveling this same road.

Let’s go in reverse order: first, the switch from MXMS to MSM.  This is fairly standard stuff, and for the most part is detailed in MSM’s POD. 

  • Change anything like so: method foo (Str $class: ) to method foo ($class: )Method::Signatures doesn’t believe in type checking invocants, and thus neither does MSM.
  • I had several unit tests that were verifying that methods didn’t accept bad parameters by checking the contents of the exception that was thrown.  Since MS / MSM fixes MXMS’s hideous error messages, I had to change those tests.  I still haven’t decided if it was bad of me to rely on the exact exception text in the first place or not.  The reason I did so, though, was so that a completely unrelated error wouldn’t pass the test, which would be bad.  So I left it that way for now.
  • I had to mark some slurpy parameters as optional (although I think I went back and fixed this in MS / MSM later).
  • I changed parameter lists like @args to @_.  MS / MSM allows that latter signature to mean “leave my arguments alone, just shift off $self for me.” I could have left them as @args and they would have worked fine, but I figured, why have the unnecessary array copy?
Nothing significant, really.

Next, the Moose upgrade.  I can’t find anything in the logs that indicate that I did a single thing for this.  All our existing Moose code Just Worked.  Of course, Moose is a relatively recent thing for us, and it still doesn’t constitute a major percentage of the total codebase.  So perhaps we just aren’t doing anything Moosively interesting enough.  I seem to (vaguely) recall having to upgrade some MooseX modules to match the new Moose versions, and maybe having to wait on a few of those to catch up, but that’s to be expected.

Finally, the actual Perl version upgrade.  I didn’t really expect to have anything to do for this, but it turns out there were a few things here and there.  Remember, our codebase goes back to 2001 ... at least, that’s the earliest commit I see in the VC logs (currently Git, converted from Subversion, converted from CVS).  So there were a few long-standing bugs that the good Perl folks have decided to expunge, and I can’t really blame them for that.  But I’ll call them out here, in case they should prove to be pitfalls to you too, if you’re ever porting anything from the general vicinity of 5.8 to the neighborhood of 5.14.

Switch structures. We had some code using use Switch 'Perl6'; I had to change all those lines to use feature 'switch'; I thought that would be all it took, but apparently the Switch module allows you to omit parends for when clauses (most likely ’cause Perl 6 allows you to omit ’em), whereas the new core switch says you need them.  Still, not a huge quantity of changes, and easy to search for.

qw in foreach loops. Here’s an interesting one.  Apparently you used to be able to do this:

foreach my $id qw(timestamp person_id property_id unit_id) {
which I never even realized.  I can’t imagine that I ever would have tried it, personally.  It just looks wrong to me.  To me, that’s an unparenthesized foreach list, and that’s not allowed.  Apparently someone over in Perl development land agreed with me, because you’re not allowed to do it any more.  So I just changed all those instances to something like:
foreach my $id (qw< timestamp person_id property_id unit_id >) {
Changing the qw delimiter to something other than parends wasn’t strictly necessary, but a) I think it makes it a bit clearer as to what’s going on here, and b) I just don’t like using parends for qw.  We use parends all the time, for loads and loads of things already.  I want my qw constructs to stand out a little more.  My choice of angle brackets is mainly because that’s what Perl 6 will use (without the actual “qw,” even), so I feel like I’m somehow getting ready for that change.  Probably I’m fooling myself.  But, hey, I had to pick something, and that was as good a choice as any.

This was a big pain in the ass to search for, as it turns out.  It was okay as long as the foreach and the qw were all on the same line, but not so much when they weren’t.  I missed a few the first time around (and the second time around, to be honest), but luckily we have a very diligent QA team, and they had my back.

Named subs in Mason templates. Okay, first of all, let me categorically state that I don’t care for Mason.  I’m sorry if it’s your favorite templating system or whatnot, but after working for 5 years in 2 million lines of code that use it, I just find that it not only allows tainting your presentation layer with back-end-specific code, it practically encourages it.  Yes, yes, blah-di-blah, you-can-write-bad-code-in-any-language/toolset, but I’d rather my language/toolset made it hard for me to write bad code, not reward me with candy and kittens every time I do.

So, this problem, which we only had one instance of, involved having a named sub in the Mason template.  Apparently, this produces one of those weird closure warnings: “variable will not stay shared,” I think it was.  This has to do with how Mason eval’s the template and all that happy horsepucky.  Anyway, solution was just to change the named sub to an anonymous sub and assign it to a variable, and then call the sub via said variable.  Simple enough, once I understood the problem.

I’m not sure if this is a new warning (or, rather, a new instance of an old warning), or if perhaps the newer version of Mason just wasn’t willing to overlook the warning where the old one was.  My brain hurt so much by the time I got this far that I just wanted to fix it and move on with life.  (And, by the way: the mere existence of an entire subroutine in the middle of a Mason template just goes to prove my original point about Mason, as far as I’m concerned.)

UNIVERSAL isa. So, apparently, code like use UNIVERSAL qw( isa can ); is deprecated now.  Whereas it used to be the recommended way to do things.  See how times change?  I’m not entirely sure what’s going on here, but I’m sure has to do with esoteric Perl OO guts and I probably don’t really need to know.  Mainly this involved changing a lot of code like this:

unless isa($validator, 'CODE');
to look like this instead:
unless reftype($validator) eq 'CODE';
and of course adding reftype to the import list for Scalar::Util.  There was also a bit of changing this:
if isa($obj, $class);
to look like this:
if $obj->isa($class);
and one instance of:
unless can($handler, 'new');
which had to change to:
unless $handler->can('new');
Wrap-up. And that was pretty much it.  Overall, not a bad set of changes to have to make.  My (professional) programming life started out in C and C++, so let me tell you this is nothing when it comes to porting.  Still, a bit more than some Perl porting projects I’ve worked on, so I thought I’d share.  Hopefully someone will find the info useful.


Great writeup.

At various companies where people are complaining that they cannot upgrade their perl... it would be awesome to have a mapping of what code needs to be changed - probably based on the perl delta documents - or even some code that would recognize constructs that were deprecated or changed between versions of Perl and might even suggest fixes...

Just thinking aloud without being able to back it with tuits :)

Interesting thanks.

Having identified the kind of changes that were needed, we're you able to automate any of it? If so I'd be curious to know what approach you took.

Thanks for the writeup - I enjoyed reading it (found it on an issue of Perl Weekly).

Thanks everyone for the comments, and thanks especially to Gabor for the nod (once again) in Perl Weekly.

> Having identified the kind of changes that were needed, we're you able to automate any of it? If so I'd be curious to know what approach you took.

No, we didn't attempt to automate any of this. Remember, this was a 10-year-old codebase, very large ... the total number of problems we ran into was quite small, considering. And all of them were trivial to search for using grep/ack, except for the qw one. And by the time I realized I wasn't finding them all, I was halfway done, and it didn't really occur to me to try to automate it at that point.

It's also very interesting to ponder if any sort of automation would be useful for people porting from (or to) a different version than I was. Probably it would need to be something like Gabor ponders above: something that knows about the different Perl deltas, can apply the right set of changes, etc. Also almost certainly it would need to use PPI instead of stupid-simple grep/ack searching. And consequently it would take forever. Although perhaps you wouldn't care, if it worked well enough.

Anyways, like Gabor, I don't have the tuits right now. But maybe all this pondering will inspire someone. :-)

Mainly this involved changing a lot of code like this:
unless isa($validator, 'CODE');
to look like this instead:
unless reftype($validator) eq 'CODE';

why not go the whole way and change it into

if reftype($validator) ne 'CODE';

I'm not a PBP#66 fanatic and find it really hard to decide if if ! defined() is better then unless defined(). 'unless defined' seems to be more readable then the if-construct. But in this case I'd argue that reading and grokking the code is improved by switching to 'if'.

Leave a comment

About Buddy Burden

user-pic 14 years in California, 25 years in Perl, 34 years in computers, 55 years in bare feet.