Perl and Me, Part 11: Please Mr. Perl, Will You DWIM?
This is part 11 of an ongoing series where I explore my relationship with Perl. You may wish to begin at the beginning.
This week we look at trade-offs in expressivity.
Last week I told a story about a CS professor who was inexplicably afraid of recursion. Other than the obvious moral of “don’t be afraid of things that use underlying complexity to make your external code simpler,”1 what exactly does this have to do with Perl?
We’re programmers. Nearly every single decision we make involves a trade-off of some sort. This way will be more efficient, but harder to maintain. This algorithm will be faster, but use more memory. This piece of software is more widely used, and therefore easier to find people to work on it, but feature-wise it sucks ass and can’t hold a candle to this other piece of software, which is dead brilliant but almost entirely undocumented and nobody’s ever heard of it. If we upgrade, we risk introducing bugs and downtime from weird backcompat issues; if we don’t, we risk becoming outdated, insecure, and left behind in the marketplace. There’s always some trade-off, and it’s our job to pick one. We don’t always pick the right one. In fact, most of the time there isn’t really a “right” one. What seems right may end up being wrong, and what’s actually right today may end up becoming wrong tomorrow. But we do the best we can. And when the benefits outweigh the consequences, that doesn’t make the consequences go away. It just means we have to learn to live with them.
There’s a price we pay for Perl’s DWIMmery. And the price is, sometimes Perl gets it wrong. But, you know what? That just doesn’t happen that often. If it did, I wouldn’t be sitting here after 17 years rambling on for eleven frigging weeks about why I love Perl so damn much. It happens, sometimes, occasionally, and, when it does, I just deal with it, and move on. That’s all. I don’t stop using $_ just because I’ve spent a few multi-hour sessions trying to figure out what in holy hell happened to my loop variable. Some people do, of course. I’ve met a few, even. “Oh, I never use $_,” they’ll say. “Too confusing.” And I always think the same thing: uhhh ... hunh. Well, more power to ya. As for me, I’m going to stick with the syntax that helps me Get Shit Done quickly and efficiently and with a minimum amount of unnecessary and redundant code.
And Perl gives you that. It does some clever shit. Sometimes it’s so clever that it fools even you, and you wrote the damn code. Many coders will say they don’t like it when code is too clever. Well, maybe so, but what’s the alternative? Code that’s too stupid? Was my 200 lines of Pascal2 likely to be clearer and easier to maintain than my original 25 lines? You know, I don’t think it was. It’s true, the shortest amount of code is not always the best. It’s possible to make your code so concise and compact that it becomes dense enough that even the reader’s brain cannot escape its specific gravity. But let’s not forget that more code is not always better either. Who wants to have read and decipher 1,000 words when 10 would have done just as well? At the risk of going completely off-topic,3 I will quote the eminent Lewis Carroll:
“I quite agree with you,” said the Duchess; “and the moral of that is—‘Be what you would seem to be’—or, if you’d like it put more simply—‘Never imagine yourself not to be otherwise than what it might appear to others that what you were or might have been was not otherwise than what you had been would have appeared to them to be otherwise.’”
So there’s always a balance we strive for regarding verbosity, and I believe Perl strikes that balance for us perfectly. And that it does so by liberal application of DWIMmery. Taking the DWIM out of Perl would make it not-Perl, in some fundamental way, and that’s what really bugs me about the whole smartmatching debate. “It’s too magical,” is often just another way of saying “I don’t fully understand how this works and it scares me.”4 Don’t be scared. Let it do its thing. Will it eventually bite you in the butt? Of course it will. But you have to look at the trade-off. How much expressive, productive code are you going to write vs how many times are you going find that it goes off in the weeds and does something entirely wrong? My experience with $_, and with curly braces, and with Moose, and my experience thus far with smartmatching, tells me that, in the end, I’m going to come out on top.
I mentioned that I continue to use Devel::Declare-based modules, even though they’re currently out of fashion. In fact, I use three such modules (MooseX::Declare, Method::Signatures, and TryCatch) very heavily, and all three make very heavy use of DD. When I say “very heavily,” I mean “nearly every day”—remember that I write programs for myself a lot. I have exactly one problem with these modules, although I freely admit it’s a bitchy one: it throws off the line numbers. That means what when you get an error, and that error is after some code that DD fiddled with, there’s a chance that Perl will report the error as being on the wrong line. How big of a chance? Well, that depends on a lot of factors. The chances aren’t very high if you do everything just so. But they increase with the more such constructs that precede your error, and most especially with how many lines those constructs take up. If they never have any newlines in them at all, you’re good. But I have a tendency to do things like stretch my method signatures and class declarations over multiple lines, sometimes even sticking comments in there. And I buck the typical Perl trend of putting opening curly braces on the same line as the thing that introduces them.5 So I probably hit this problem more than anyone else, and it really irks me. It even irked me enough to try figure out a patch for it, which involved deep diving into the guts of the code that you may recall mst has proclaimed to be “a big bag of crack” (even though he wrote it). At the end of the day, I think the problem probably relates to the fact that DD uses Perl core code to skip whitespace, and it’s tricky to know how many newlines that skips, and/or to communicate that back up the many layers of modules to the ultimate client for them to be able to do something about it. In the end, I think I might have to make a small but fundamental change to DD itself, plus probaby patch the things that are actually using DD to figure out where the line numbers are getting off and fix them. Or else I’m completely off and I don’t understand what’s going on at all, which would probably be worse. So it’s definitely what you’d call non-trivial. I may roll up my sleeves and wade in there and do it anyway, one day, if I get inspired.6
But, in the meantime, I just live with it. It doesn’t come up all the time, and, even when it does come up, most of the time I can figure out where the error actually is just by looking at it. For those rare cases where it happens to come up, and it happens to be far enough off that the general neighborhood isn’t sufficient to give me a clue, and it happens to be an obsure enough error that I can’t just grep for it and find it even without the proper line number, I just made a macro for my editor to sprinkle some #line directives around,7 and another to clean them up afterwards. It usually costs me about 4-6 extra keystrokes, once or twice a month. From my perspective, that’s a pretty small price to pay for being able to write clear, concise code.
Of course, that’s me. I already discussed at some length why code legibility is a pretty important thing to me. Others may decide that this is not a good trade-off for them, and that’s okay too. Everybody’s different, and that’s what keeps the world an interesting place, I always say. I just think it behooves us to be aware of the particular trade-offs we’re making.
For my part, I’ll take the shorter, more concise, more legible code using $_, and Moose, and smartmatching, and my various favorite DD-based modules, over being more verbose and convoluted in my code.8 Many of you will retort that they don’t find code using, say, smartmacthing clearer. You may point out that, when you see the smartmatch operator (or the when keyword), you now have no idea what it actually means because it’s so complicated and DWIMmy. But I say that’s because you’ve decided you don’t like the concept of smartmatching, and you’ve become convinced of its complexity. The truth is, 90% of the time you see ~~—probably more—you know exactly what it does without even thinking about it. The remaining 10% of the time you can look at the table, and you won’t need to read the whole thing: just the one relevant line.
Look, people are going to hate new things sometimes. I hate new things sometimes. When I do, I just don’t use those things. Sure, I can’t avoid them entirely, because other people will use them, and I will have to read or even maintain that code. This is part of life. People use words I don’t like all the time too, and they take actions I don’t like, and they pass laws I don’t like, and they make hideously crappy reality TV shows that I absolutely detest. That’s just the way the world works. I been living here on this planet for coming up on half a century now, and I’m used to this sort of shit by now. There: I just used a word that some of you won’t like. You can stop reading my blog posts now if if bothers you.9 But please don’t try to stop me from using that word. Don’t take away my freedom to use whatever word I like.
I guess that’s what bugs me about the whole smartmatching thing. I feel like something sorta nifty is being taken away. Perl is all about DWIM, and all about TMTOWTDI, and TMTOWTDI is just another way to spell “freedom.” Maybe losing smartmatching isn’t that big a deal, but it’s the general attitude that vaguely unnerves me. The idea that some folks have decided that Perl is too ... whatever. In this case too DWIMmy. So Perl is too this or that and must therefore be reined in. For our own good. I don’t dig that.
I have no idea where we’ll go next week. Maybe I’ll actually start looking for some conclusions after all.
1 Which is a pretty good moral all by itself, now I consider it.
2 From my college assignment that I discussed last week, I mean.
3 And, yes, I’m sure there are many of you clever-clogs out there who just yelled “too late!” Rest assured that I sufficiently appreciate your wit and humour without the need for adding comments to that effect.
4 Note my specific use of the word “often.” You, obviously, are one of the exceptions to that rule.
5 I do this because I’m one of those eccentric people who likes their open and close curly braces to line up vertically. Maybe it has something to do with the fact that I was a C++ programmer before I was a Perl programmer. But I don’t really want to debate matters of style; the point is, this is Perl, and TMTOWTDI, and I should be able to program in the style to which I have become accustomed.
6 Assuming DD isn’t rendered completely obsolete by Perl’s new keyword API first, which is actually far more likely to happen.
7 See “Plain Old Comments (Not!)” in perlsyn if you don’t know what #line does.
8 And, yes, I’m aware of the irony that I like my code to be concise and elegant while my prose tends to be meandering and tangential. Another thing you needn’t point out in the comments.
9 Although, realistically, if you were inclined to stop reading my blog posts because I used that word, you would have been gone long before now.
14 years in California, 25 years in Perl, 34 years in computers, 55 years in bare feet.
I think that the reason C/C++ programmers put the braces on a new line is that you can omit them in some circumstances. You can't omit them in Perl the same way, so there is no need to make them stand out.
Smart match has some problems which are mostly because it came from Perl6 where they have a more specific type system.
Well, as I say, I like my braces (and other matching parend-like punctuation) to line up vertically. That's just easier to read, to me.
Possibly. And it's still possible that smartmatch will be retained, only in a simplified version that will be mostly backwards compatible with what we have now, at least if you're sticking to the simpler uses of it. But that's not what I predict. I have no concrete evidence whatsoever, I'll freely admit, but my gut feeling is that we've created such a negative perception of it that it will disappear entirely, most people will claim victory, and evolution will be stymied. By itself, that won't mean much; we'll just have to see if that's an attitude that takes hold and becomes more prevalent. If so, I think it will be the thing that finally kills Perl. (Or at least Perl 5. Perl 6 is a whole 'nother kettle o' fish.)