3 features I would like to see in Perl
A few days ago I read Features Perl 5 Needs in 2012 by chromatic and while I thought the ideas were nice the only one I really cared about was a replacement for XS. I have tried with XS and FFI to bring in new libraries and it is just so painful. Marcus believes that Perl garbage collection needs a serious overhaul and I agree. Improved gc in a language is one of those things that helps ease development pain, specifically scalability issues. Java is always improving its gc to better meet new performance requirements. Structured core exceptions is another improvement that would be very nice. This got me thinking what would I like to see in the next version of Perl. I have never contributed to Perl core, never written a Perl book, and I do not follow p5p or Perl6 development. I have looked a little at the Perl source but nothing serious. This makes me an outsider to Perl development so I have a different viewpoint on what I should get out of Perl.
The number one thing Perl needs is speed. It needs to be faster to stay competitive. This is one of the reasons that people say Perl is dead, no news about large scale performance improvements. The python community has PyPy and Psycho as performance options. Facebook developed HipHop for PHP which gives a 3-5x speed boost with software like Drupal. The Perl community has B::C which can improve program startup time and B::CC which may improve runtime speeds. Reini Urban has covered this material on both the Perl 5 and Perl 6 front.
Now I don't want to get off on a rant here ;)
I read articles like Perl 5.16 and beyond and the whole focus is new syntactic sugar and how adding a MOP to Perl which will make Moose faster. The only mention of something becoming faster is by adding all new features to Perl and that only makes certain things faster. What about the rest of us? I try to keep up the good tenets of functional programming in my applications and I have never had the need to use Moose in an application. I have literally dozens of Perl programs on my computer that I use in a day to day fashion some I wrote, some I didn't and none of them use Moose, Mouse, Moo, etc.. I want improvements to Perl that make my existing applications faster.
C was initially developed between 1969 and 1973, making it 40 years old. Java 1 was released at the beginning of 1996 making it 16 years old. Perl was first released in 1987 making it 25 years old. GCC and clang for C and the Oracle JVM get faster. If you take newer versions and test against older versions with the same code there is a trend for the newer compilers and JVM to have much better performance. The same cannot be said for Perl. Perl::Formance is an ongoing project that tracks the performance of Perl versions against different benchmark programs. The general trend is Perl performance is a plateau and gets slightly worse in some cases.
This performance and optimization stagnation Perl is in does nothing to help expand its market share and improve its image. The reality we all already know is that there is a finite amount of programmers in the world and Perl is vying for position against the constant influx of new languages as well as competing against existing languages. Some may consider the community staying the same size a good thing but my point of view on the matter is the community is either growing or dying. CPAN may seem magical to some but it is just a group of developers who choose to build libraries in and for Perl. For CPAN to stay vibrant it needs to add modules that meet the ever expanding needs of the programming world. An example of an under represented segment is graph database support. The PHP and Java community have us beat bad in those areas. Why? One of the reasons is both those languages have tools that allow them to scale far beyond the base implementation which is an important scalability concern.
I can take a PHP application that is slower than its Perl equivalent, and without changing the code compile it with HipHop and now it is faster than the Perl version. Does this work the same with B::C or B::CC? No. This is the end of my rant.
The top features I want to see in Perl:
1. A faster perl across the board
2. Better gc which would probably make applications faster
3. XS, FFI improvements to allow faster support of new libraries in the wild
Everything else for me is unimportant. People still use C because it is fast not because they are constantly adding new features to the language. On a side note I just saw a tweet from chromatic stating he has an idea of how to do better exceptions in core and stay backwards compatible.
I like writing Perl code and since most of it is open source I might as well talk about it too. @KirkKimmel on twitter
A better GC likely won't improve performance much in the average case. Reference counting is about as cheap as it gets, at least until you have a highly nested structure you want to free.
With that said, moving the reference count out of the SV header could give memory improvements in shared memory situations such as a preforking server.
Code compiled with B::C is has about 10x faster startup-time and uses much less memory.
But run-time is not faster, yes.
With B::CC I still have the problem of stack-smashing, but I think I had an idea last month to fix that.
For general optimization ideas p5p is not the right address, unfortunately.
> The number one thing Perl needs is speed
Fully agreed.
> The number one thing Perl needs is speed. It needs to be faster to stay competitive.
I think you need to be careful about over generalizing here. Personally, my workloads are almost entirely IO bound and I have a feeling that's a very common situation. I find Perl's performance perfectly adequate for this. Thus if I had the choice between a new feature I would use and a performance increase that wouldn't affect me, you can bet I'd choose a new feature.
I agree with "speed." Perl is no longer the fastest language even for the "simple" text processing where it excels, and I have two stories from work to back that up. (And, addressing cbt's comment, both ought to be I/O bound...)
First was a suite of nawk scripts that read tabular text input and spit out dozens of little XML files. It even hit nawk's compiled-in limit of 20 filehandles at one point, so we reworked the process to get around it. I thought it was silly to still be using nawk when Perl is cleaner, faster, and has no arbitrary filehandle limits. Well, after a day of rewriting and benchmarking that little script, either with the same structure or "improved" logic/control flow, I could not match or beat its performance in perl! Perl being slower than nawk for text file processing is a sign that Perl has strayed quite far from its roots.
Another example was a perl program that merged a few hundred text files, each about a gigabyte, into a single text file of about 60Gb. There were two keys to getting the fastest run time out of our 4-core, net-attached-storage boxes.
First was adjusting the size of the read & write buffers. Perl's built-in read buffer is set at 4k, based on some early 90's benchmarking, and is impossible to change on some platforms without recompiling. (Not even sure if you can change the write buffering). I know about read, sysread, CPAN modules but didn't want to implement in Perl what really should be achieved by tweaking an internal setting.
The other trick was threading. I "use threads" sometimes and fork other times, but perl's threading still leaves much to be desired.
I re-wrote the entire program in C#, was able to tune the buffer sizes, and saw dramatic performance improvements. I did try backporting what I learned to the perl code, but could not get far, and didn't have the mandate from work to continue once the C# was ready.
Have you taken a look at the new FFI::Raw module?
https://metacpan.org/module/FFI::Raw
It comes with a lot of examples:
https://github.com/ghedo/p5-FFI-Raw/tree/master/eg
This makes it hard to read even well-written code of programmers who happen to use features you are less familiar with.