Hey Nelson, or, Perceptions
Hey Nelson,
While I’m delivering the news, here’s something for you ignorant American backwoods motherfuckers. Some people’s names have “special characters” in them. Like François Rabelais or Björk Guðmundsdóttir or 艾未未. It’s 2011; the only software that can’t handle Unicode properly is Perl. (As if you needed another reason not to use Perl.)
please quit being an ignorant backwoods motherfucker and stop talking shit about crap you don’t know anything about.
To the Perl folk reading this — the problem we’re dealing with in terms of perception nowadays is confirmation bias. Nelson hates Perl, sees one question on StackOverflow that is making the rounds because Tom answered with one of his obsessively detailed (and therefore huge) missives, generalises wildly from a shallow read of the QA, and then – surprise – finds his opinion further confirmed.
I don’t know that there is a way to get out of this bind. Reasoning is flawed because it evolved as a means to convince others, not to figure out the best decisions to make. This is why we are addled with so many cognitive biases. That hypothesis also explains why people slip entirely unrelated jabs (or memes in general) into some other argument they’re making, as Nelson did there: a bonding ritual for the likeminded.
Finding a community that largely has no prior opinion of Perl (which likely means it has cohered around something other than programming) might be the only way to overcome this. Cf. BioPerl as an example.
Waxing philosophical
Of course there is another option; press releases. I realize that every script written to assist the busy sysadmin will not merit a press release, but there are many applications of Perl which do. I find many people do not know what Perl can do because they have not been told and are too busy to go look. It happens to all areas of IT and computing. For instance, the Oracle DBA that doesn't know what nvarchar2 type is or what it is for.
Why, it might be useful to suggest to Perl programmers what features they should list on each of their scripts and apps. Character set support would be simply one of those features. It won't be long before many people are thinking that perl does almost everything rather than almost nothing right.
I know that is kind of like mandating documentation but perhaps some clever sod might go ahead and write a utility that helps determine features that are supported by a Perl application and give a nice pretty listing of those.
I'd be pretty damn happy if that was a Padre plugin actually.
I'm not entirely convinced the perception is much of a problem. The language will remain useful as long as enough people are around to keep the community alive. That seems to be the case regardless of the already prevalent negative perception. And I suppose to some extent it's a bit of a competitive edge to be underestimated, but that's probably only a marginal advantage. Sure, it feels really nice to be the shining star, but that time has passed. It may come again, it may not. But the important thing to me is that the language remains useful.
I look at it this way: your work I know. Nelson, I've never heard of and given what you quoted, I see no reason to learn more. Things like this are one of the items that puzzle me more and more as I see the phenomena both continue and increase. What is it that caused hatred in reference to a programming language? I've thought about it for a very long time and can come up with no rationale that seems sensible. If this were one group hating another then the usual excuse of insecurity would come to mind. But this is hatred of a computer language for God's sake---how in the world does that (forgive me Spock) compute?
From what I understand, Perl actually has very good Unicode support even when not compared to other languages. But compared to other popular languages, like PHP and Python, Perl's Unicode support is outstanding.
Shimshon: you are right and that is the reason for the first part of my response.
Hugh: Nelson built the old Google search SOAP API (since disabled) was his project when he worked there after graduating from MIT and has generally worked on distributed systems in various shapes and forms, incl. a startup. He was one of the early commentary-on-everything bloggers.
Carey: it’s only a competitive advantage when it also means that interest in the langauge is the mark of skilled programmers who are driven to IT by more than just money. For Perl, that is sadly not the case – there are a lot of weak developers hogging the space while demand for skilled developers far outstrips supply. While the situation is not fatal, it is far from advantageous. And the way things are does mean that it is harder to earn a living with Perl than it should be.
“Mr. Z”: press releases won’t work if they fall on already closed ears. As for the rest of your suggestions, the actual problems are already known and are slowly being worked on – faster in the last 2 years than in the rest of the decade before. (No, they do not include “does not support Unicode”. Support has existed for a long time, it just had some problems due to confusion in the original implementers. They are mostly cleaned up now thanks almost singularly to the heroic efforts of Karl Williamson. Perl’s Unicode support is now more sophisticated than that of most other languages.) Nelson’s jab has no factual leg to stand on, particularly considering that dealing with encodings sucks in all languages in some way or another,
His comments make me wonder if he has ever actually used unicode in any language. I certainly didn't find it easier in Python than in Perl.
And p5p is letting tchrist write docs again :(
Hey there, I'm responsible for the offending quote. Thanks for the thoughtful response. I regret my drive-by comment about Perl distracted from my larger point, which is that a whole lot of web applications (in any language) still don't handle Unicode. It's unacceptable in 2011.
I think you're right that my Perl bashing is driven by confirmation bias. I've read that post from tchrist 4 times now and each time I read it the only takeaway I get from it is "wow Perl doesn't make Unicode any easier". A lot of tchrist's caveats are issues in any language: correctly handling all multinational languages is difficult.
But there's something really limiting in his conclusion that because it's hard to do Unicode correctly in all cases, Perl shouldn't try to do any of it. That Perl programmers have to treat all Unicode as some rare and fragile beast. It's not like that in most languages.
Hi Nelson! Thanks for dropping by.
I completely agree with the primary point you were making, that there’s no excuse not to do Unicode and personal names properly in this day and age.
As for Tom – he’s smart, and he is incredibly detail-oriented, so I like to read him and to consider what he says. But I don’t always reach the same conclusions, and in general he does not represent the community as a whole, nor is everyone always glad of his… forceful style, shall we say.
Without going too far afield, the major problem with doing Unicode in Perl has been that the documentation itself was confused about the semantics of Perl’s post-8-bit-character string model. When used correctly it works great, but a lot of legacy Perl code is still subtly broken because it was written following that confused advice, and even some old parts of perl weren’t clean. So sometimes it unexpectedly turns tricky to get things clean when combining a lot of CPAN modules. Karl has been fixing the problems in the core in recent releases, though (and I can’t stress his heroic efforts enough), and I believe over time the ecosystem will catch up. What’s tricky is to avoid breaking too much code right now with these changes, so it’s taking a while and Karl has to backstep and sidestep at times.
But there are people in Perl who believe that it should be easier for the programmer, at least as far as it can be made, and ideas are being thrown around and tested. Tom’s answer on that QA unfortunately overshadowed another post that linked utf8::all, f.ex.
It’s a slow slog to shore up a language with almost 25 years of history and almost as much legacy code, plus a commitment to not break existing code if at all avoidable. But it’s getting there anyway and the rate of progress has in fact been accelerating in recent releases, which I’m elated to see.