Does Perl (5) have a future?
TL;DR: Not if it can never have a major release. Please read on.
Note: I’m now not talking about soon, or what it would be named/numbered. I’m talking about ever.
Let me tell you my story, some of you may know it. I’m a Ph.D. candidate in Physics. Programming is woefully ignored in science education now. All my professors learned FORTRAN in courses during their Ph.D. (or B.S. in some cases) but now, given the ease of Mathematica, we seem to be expected to pick it up as we go on.
One day I had a nutty idea, I wanted to parse one of the temporary files created during LaTeX compilation. A friend, who knew next to nothing about Perl, suggested I needed Perl and Regexes. I can here you out there, saying “now I had two problems”. Not so. I learned, I improved and finally my first Perl script was out in the world. It doesn’t bear much resemblance to its current form but it worked.
This project had nothing to do with science, but I was awed by the power I now possessed. I went to start using Perl in my work. Perl finally solved a near-intractable programming problem in Mathematica with relative (hmmmm) ease. Physics::UEMColumn now forms the basis of my Ph.D. thesis which I am defending this spring.
To get that simulation working I needed more tools, so I wrote them, and those tools needed tools so I wrote them and suddenly I discovered that I had something I could give back, so I’m writing it too. Further I often need PDL, and amazing numerical package for Perl. It really needs some internal TLC, which David Mertens, project pumpking Chris Marshall and others are planning.
Why am I telling you this? Because in 2009 I became a Perl programmer, and I fell for language hard. I love Perl. I came for regexes, but I stayed because it works well with the way I think. I use it for lots of things now. Unfortunately few others have come with me.
Chromatic implores us to write good software and show it off. For these years I have tried, and it doesn’t work. I have given talks at my university, I have told friends, I have used it to help coworkers, I share with the community. I have recruited one person. I have converted none. Most people who know Python or ROOT (C++) laugh at me. Its not enough.
Until yesterday, I had a steely resolve. Even in the face of Stevan’s recent pronouncement and embarking on replacing the Perl internals with Scala, which would probably kill or at least maim most of my work. Moe wasn’t the future of Perl, just an offshoot, right?
Yesterday, for the first time, I saw a flaw. There are people out there who really think that Perl will never see another major point release. I don’t mean soon, I don’t mean for cheap marketing, I mean ever. I mean for at some point breaking some small amount of backward compatibility. For some the reason is that compatibility is too important (the future is our past), for some its that Perl 6 is the future of Perl 5 (whether spoken/believed or not) and some its just because there is no number available. All that got me to thinking …
Is the work on Perl 5 really just a shim to support CPAN? Is it really going to be an ever-growing feature pragma? Will there always need to be a magic incantation to activate some of the best Unicode support in the programming world? Will new Perlers need to be told to use good practices like strict and warnings forever? Is the herculean work of Nick Clark and Dave Mitchell just going to go for naught after a few years of a usable Perl 6 and Moe?
If Perl can never have a major point release. I think all of the above is fated to be true. I’m not asking about soon; I’m asking about ever.
In my mind I’m planning future work on the Perl we now call 5. I want to help improve the giant XS extension called PDL, possibly using another one called Prima. I intend to keep working on Alien::Base. But is it all just going to vaporize in a few years do to lack of external interest, or after being replaced by something? Why should I?
In the upcoming months, I will be looking for my first real job. It might be a post-doc appointment with Pacific Northwest National Lab (PNNL) doing the hardware/software integration for a brand new ultrafast electron microscope. Should I do it in Perl? I will only be there a year or two, is it fair to saddle them with that? Will they find someone else who can maintain it?
If I don’t do that, I might form a company, one that aids research groups in writing the software that they often cobble together, for analysis, data warehousing and search, for hardware integration. Would I do that in Perl?
If the powers that be don’t see a major point release at some point in Perl’s future, then it has none. The userbase will keep sliding away. Not purely for a number, or marketing, or perception, but because you will need too much knowledge to start using it. You will need to know the incantations to do things right, ones that we already know, but new users can’t find. How many people find a CPAN module that does just what they want, and so set out to write their Perl first script, only to fail and walk away? We can’t ever know.
I ask you p5p (whom I admire greatly) and I ask you Larry, creator, benevolent dictator, does the language we call Perl 5 have a future? At some time, will we see a major release? If not, then I don’t know what I’m working towards. Should I tie my future to this language?
As I delve into the deeper Perl magic I like to share what I can.
Please note that my closing line is not a threat to leave the community. Its commentary on what I perceive as a problem facing my future career.
Joel -
Thank you for articulating these concerns so well. I love Perl and I’ll use it in my own work because it is such a wonderful glue language and I love the community.
But it would be REALLY NICE if the powers that be gave me a little boost by making the language just a little easier for others to join the party.
Do you love Perl (mostly rhetorical)? It is obvious you do.
I have only been using Perl since 1997. I have a few CPAN distributions. Some of them are useful, some are not. I love Perl. I write nearly everything in Perl. My job for the past 16 years has been to write Perl. We hire people to write Perl. We prefer our company acquisitions to also be writing in Perl.
I have watched and waited with baited breath for Perl 6. I have followed it from the beginning when Chip was doing the work. I have waited for the proverbial Christmas to come. It has obviously not come yet.
I have also watched RedHat attempt to systematically kill Perl and really all of the dynamic languages with their “stable” enterprise releases. Too much of the world is still stuck on ancient Perls because of the incredibly poor management at RedHat. More than anyone, I would lay the blame for the fall of Perl at the feet of the people at RedHat. But they are only one part of the problem.
I understand that nothing will change. I understand that Python will continue to be more sexy to academics (look ma, it writes itself). I understand that Java will continue to be taught as core CS curriculum. I understand that Microsoft, and Apple, and Google, and the next corporation will continue to push their own walled garden. I understand that Perl will continue to be marginalized by the inadvertent actions of many parties. I understand that my opinions matter little except to the small portion of the world to which I am bound.
Be all that as it may, I am in the fortunate position of receiving a salary to write Perl during the past 16 years and with every prospect of continuing on for at least the next 10. And not just any Perl, but blasted ancient Perl 5.008.
So should you write Perl? Is it worth your time? Will you have job prospects rolling in? Is Perl a dead end? I don’t know. Will having a major release either Perl5 or Perl6 change anything whatsoever? I doubt it.
But if you love Perl, write Perl! Your opinions and abilities apply to a small and different portion of the world - but they matter! If you love something, do it, and share it, and forget what the “small world” using-the-other-language-of-the decade next door thinks! Perl only dies if people stop loving it. That will not happen.
Joel,
For the record, Moe is just a thought experiment and Scala is just the language I chose to do the experiment in, I do not intend long term to replace the internals with the JVM.
As far as it killing or maiming your work, while there is very little chance that XS will be preserved with Moe (or with any of the other “new” Perl 5 efforts out there such as p2), you can bet that linking to a well organized C library (which I assume PDL and Prima both are) will be supported in some way. And I suspect that your work on Alien::Base would not be lost either, perhaps a little maimed but nothing unrecoverable.
@Paul I will continue to write Perl, because I do love it. I don’t worry about the popularity of other languages, other than, if I cannot find a client willing to pay for new Perl code, then I cannot write Perl code.
@Stevan, I do know that Moe isn’t really a threat and I applaud you for doing it. I was simply overwhelmingly saddened yesterday to see a small but important subset of Perl people dismiss any idea of a future major version of Perl. It makes me think that to really forward the language perhaps Moe is needed. But in this case working on an XS project is a wasted endevour.
Prima I believe is well organize C other than the XS layer, though I think it is somehow Perl specific. PDL on the other hand is a massive XS generator (yeah, I know). In the end a C JIT or a real FFI (p2 or David’s work with TCC) are possibly a better future.
I really am asking a question. Is it worth sticking with this Perl 5? I’m at a life junction.
@Joel - You have now asked two questions: Does Perl (5) have a future? and now you have asked: Is it worth sticking with this Perl 5?
Does Perl 5 have a future. Yes it does.
Is it worth sticking with Perl 5. For many Yes.
So to refine your question to: do you have a future with Perl 5? That depends on if you are offering something marketable to somebody who wants it (even if the only somebody is you). If you were interested in relocating, we (and many other firms) hire people that write Perl. But that assumes you want to write what we need you to write. If you want to write what you want to write, there isn’t anybody reading this thread that can answer the question “Do you have a future with Perl 5.”
Perl 5 will be as viable in 10 years as it is now. Determining its viability now is your own subjective call. If you are writing your own project to try and scratch somebody-elses itch, likely no language will be viable.
I used to play a video game called Subspace, created by the now-defunct Virgin Interactive Entertainment (VIE). I first played this game in open beta in middle school. It was one of the first MMOs that wasn’t an RPG.
Later, after college, I started playing it again. VIE was dead, the game source was lost, the game client was reimplimented, the server code was continually hacked to fix bugs (VIE gave away the server with the full CD).
And for the next six years I played as the population slowly dropped. We all knew that without a steady influx of new players to replace the old ones, the game would die, but none of us knew how to fix it.
We tried a lot of things. I learned Java, wrote bots to make new ways of playing the game. My zone, A Small Warzone, died. Over the years I tried twice to bring it back, to no avail.
The game died because it was old and nobody was actively developing it. Without the steady flow of new users telling their friends and in general being loud about this cool thing they found, the population could not sustain itself and it went extinct. I imagine there might still be a few people dogfighting amongst the asteroids, but I’ve left them behind.
Perl is not a video game, but community is even more important for programming languages. Bumping the major version is a marketing/publicity stunt to be sure, but marketing/publicity has always been our Achilles heel.
I’m not saying Perl is dead. I’m not saying I’m going to stop using it, ever. I am saying that it is dying and will continue to die unless we improve our perception.
Yes, this means building cool shit in Perl. We’re all doing that. We’re all even talking about it, or else what is blogs.perl.org for? It means local outreach, the Perl Mongers, bringing new users in (7 times out of 10 I have to tell those new people “No, Perl 5 and Perl 6 are different languages, and you want to learn Perl 5). It means writing more and better code on CPAN so that every framework and library the other languages have we can say we have too.
But it may also mean breaking away from a past that grows more distant and less relevant to the new generations of programmers.
Choosing very specific bits to comment on:
a) You can use ROOT from Perl. It works quite well.
b) if you think writing XS is a lost cause before there’s a perl implementation even on the horizon that doesn’t support it, then you just need to grow up.
TeX will not ever have a new major version. :-)
Nevertheless I agree with you. I just had to comment about TeX versioning…
@Paul, I agree the choice is mine on whether I should form my life around Perl 5. To do that though, I am asking for one piece of information: Is it remotely conceivable, that at some time in the future there will be a (somewhat substantial) breaking release and/or version number bump for it?
@preaction, I think we see eye-to-eye.
@Steffan, (a) yes and you made that possible IIRC. Still they laugh and point and call me silly names (ok that’s perhaps not entirely true). (b) I am being a little extreme here, its more for rhetorical purpose though (except maybe in the case of PDL).
@Alberto, yes I used the example of the definedly-fixed TeX version in a comment on another post. The fact that TeX’s version number will not change signifies that it will not ever break. This is possible because it is remarkably stable and its use-case is fully and finally set. If we say that Perl 5 is version 5 forever are we making such a claim?
@p5p, @TimToady, I would still appreciate a weigh-in.
I love Perl, too. And I can’t quite say why. It just fits my brain, I guess. I do not want to quit.
I do think that I should try to improve the community in my area, that is why we are organizing Swiss Perl Workshop 1 at the moment. This means finding local Perl people and bringing them together. Quite astonishingly, there is more Perl than you would probably think. :-) That again is why I think I don’t have to quit.
I am also waiting for Christmas, yes. I am not sure if I am young enough to see it coming, sadly. But meanwhile I can have a lot of fun with Number 5.
This is not the most elaborate post, I know. I just have a feeling that if I only write when a really big idea pops up in my head, I am almost always silent. Like if you write a blog post and mostly get no replies at all. This is by no means a rant but an observation.
And yes, it may seem contradictory that I am asking for change while arguing against change. I should clarify. I am hoping for incremental change, an acceptance of what most of us do anyway. I hope for inclusion of a mop and then something Moosey in the core.
I want these changes so that something as drastic as Moe isn’t so necessary. Indeed I think an internals rewrite would be useful, but can/will we do it at this point? I’m asking to drain the tub a little so that we don’t end up throwing out the baby. I hope you understand.
I have been using Perl since 1998. I love working with Perl, and I use it for almost all of my programming.
I have a modest proposal for the Powers That Be:
Call the next release of Perl5 “Perl7”, and rename the Perl6 code base “PerlZ”.
Numeric releases would be “production/mainstream” releases, showing the public that Perl is moving forward, while keeping the current major/minor version numbering scheme for packages that rely on knowing what versions they’re running for compatibility reasons. The major release number would be incremented whenever there is a major release.
People who want to work on porting PerlZ’s features and code into mainstream Perl# releases could keep doing so. People who want to work on the future of Perl could still work on PerlZ with the understanding that once PerlZ is ready/stable then it would be assigned a number and it would become the code base for the next major “production/mainstream” release.
Everyone wins.
This graph shows the number of Perl conferences / workshops / hackathons over time. If you look at the 2008-2012 period, there have been 127 such events in 5 years. That’s 2 per month on average, over 5 years! No so bad for the global community of a dying language.
There’s also a timeline of Perl events available. The full list is maintained in a crappy YAML file, as the comprehensive list of all Perl conferences, workshops and hackathons.
A lot of the events attract new people to Perl… or to Perl events, at least. This can be seen at YAPC (NA and EU) every time the person doing José Castro’s talk (“How to make the most of a YAPC”) asks the people in the audience to “raise [their] hand if this is [their] first YAPC”.
I’ve already listed 12 events for 2013, but that’s only the ones I currently know of. After all, I added OSDC.tw 2011 and 2012 to my file only yesterday…
Note that the list of conferences for 2013 includes two brand new Perl workshops: the first Swiss Perl Workshop and the first Polish Perl Workshop. I’m not worried about our community liveliness.
Well, you can use ROOT from Perl under certain circumstances and setups. :)
brian: I doubt it’d be that hard to get working on your Mac!
The only way I can see Perl 5 getting a new major version is by changing it’s name. The reason for that is the existence of Perl 6 and Larry’s decision to stick to that name for the language.
That only leave the renaming option open. The renaming option “Perl 5” -> “Perl5” I suggested earlier would probably only work inside parts of the Perl 5 community so we can forget about that.
At the same time we want to keep Perl as Perl, so I don’t believe in the Chocolate/Citrus/DWIM/Strawberry/… Perl strategy either, at least not as a version bumper strategy. I guess we could fork it “the C way”, i.e by naming it Perl++ or Perl# even though it feels like a cheap marketing trick to mimic the C variant naming schemes.
I love perl,joel. Please don’t forget the people who love perl.
A little background - I have worked for several companies and used around 12 languages in that time - many mainstream, and some pretty obscure ones. We are currently a Java shop - BUT - i have never let go of perl since I first started 5 years ago. I have consistently proven to others that for certain things I can do a better, quicker and more cost-effective solution in perl. People still laugh at perl, but I don’t care as I will use what works best. All of our migration tools are perl, we have a super fast file transfer protocol in perl, we have a pretty heavily used web site in perl. Our primary enterprise web app is in Java, but a lot of the back-end services to manipulate files and process videos, to securely store things are done in perl. I now have a small team dedicated to perl specific work on top of the java devs.
For what I do, v5 works. I hope perl never dies away in my lifetime - new version or not. Love perl!
HI JOEL AND EVERYONE HERE. JOEL WHAT A COINCIDENCE I’M FINDING THIS ARTICLE EXACTLY A YEAR AFTER. I HAVE A QUESTION TO ASK ALL YA EXPERTS, I’M JUST A NEWBIE AND JUST BEGUN MY CAREER AS A DEVELOPER(1.5 YRS) USING PERL. I HAVE STARTED LOVING PERL. BUT THE IMPACT AND MAGNITUDE OF PRODUCTS OTHER LANGUAGES LIKE JAVA ANDROID ARE GETTING INTO THE MARKET IS MAKING ME RETHINK BUILDING A CAREER AS A CORE PERL DEVELOPER. IS THIS THE RIGHT TIME I SHOULD SWITCH TO JAVA AND ANDROID BEFORE ITS TOO LATE. I NEED TO THINK THIS AS I AM THE SOLE BREAD EARNING MEMBER TO MY FAMILY. ALL YOUR ADVICE WILL BE MUCH APPRECIATED. THANKS IN ADVANCE
Hi, I don’t think anyone else will see this comment on an article posted so long ago. You might want to ask your question as a post of your own. Oh and don’t use all caps, that is considered impolite (yelling) in most online communities including this one.
Oh I am sorry Joel about the all caps, this was my first ever comment on any blog, thus was unaware. I will remember that henceforth. What is your opinion or view now after a year of posting the above question ?
I DO NOT get and CANNOT understand WHY WHY WHY that if Perl6 will not come will eventually led to the death of Perl????
What?
Look, I remember I started doing my first computer program on a programmable calculator in high school. I have exhausted all the APIs of the calculator and so what remains available to me are the infinite possibilities of how to solve a problem in the most efficient means. Now, when i moved the world of PC computer programming, I AM a bit surprised there. I am surprised that there seems to be an ever growing set of API and no matter how much I waited, these set of APIs keeps on growing. NOW, now this bothers me. This bothers me, because it would be difficult to create “true art” if the set of APIs keeps on changing — and displacing older APIs, etc.
WE DO NOT NEED Perl6 to come out in order to keep perl alive. Perl5 right now, is already very powerful — there is nothing more that you can ask from it. It has all the expressions that your mind needs and that is more than enough.
If you want to dig deeper and expand, then you code in C, and from there you have infinite possibilities.
The rapid expansion of APIs out there is creating a herd of powerful idiots. Baptize yourself with the first few pages of Donald Knuth’s “The Art of Computer Programming”. The most powerful engine you have is right there between your ears. yo!