Try rakudobrew and play with concurrency

rakudobrew is similar to perlbrew, but it's for Rakudo (a.k.a., Perl 6), the Perl-inspired language that we've all come to have a love/hate relationship with. I urge you to try it out, but first, some interesting new developments that you should probably know about.

By now you've probably heard of Perl 6, either the greatest dynamic programming language ever created or one of the longest-running shaggy dog stories in history (more so than even Duke Nukem Forever). That being said, with MoarVM and other experiments, it looks like Perl 6 is finally turning an interesting corner.

There's an old saying in programming of "make it good, then make it fast." They're finally working on making Perl 6 fast. In some examples, it's actually an order of magnitude faster than Perl 5. That's astonishing given that Perl 5 is already the fastest dynamic language out there. In other examples, it's more than an order of magnitude slower, but with their new profiling tools, the Perl 6 devs are quickly finding performance bottlenecks and, more importantly, finally have an architecture that they're convinced is good enough to optimize.

So, um, yeah. You've heard all this before. You've gotten your hopes up. You've been let down. Christmas after Christmas, there were no presents under that red-black tree.

So let's look at this differently. Moore's law is coming to an end. It has to be. There are simply physical limitations (they're called subatomic particles) beyond which we can't go. Until someone comes up with a revolutionary method of computation (don't hold your breath on quantum computers), we are going to see a change in how software works. If you want it faster, you don't just wait 18 months for new hardware; you go concurrent (we'll skip the distinction between concurrency and parallelism for now).

There is not, to my knowledge, a single popular dynamic programming language which has a working concurrency model. They're all broken in fundamental ways. Perl 6 aims to change that by making concurrency, if not easy, at least easier. And it's working now. In fact, even if the language is slow, if it makes it easier to do concurrent programming there are those who will adopt it (I'm looking at you, Erlang). Perl 6 is relatively easy to write and it's concurrency is easy to read. Let's take a look.

First, install rakudobrew.

git clone https://github.com/tadzik/rakudobrew ~/.rakudobrew

Then add that to your $PATH (I put mine in my .bashrc file):

export PATH=~/.rakudobrew/bin:$PATH

Then build it and (optionally) install the panda package manager (because we need more package managers, right?):

rakudobrew build moar && rakudobrew build-panda

Now test it:

$ perl6 -e 'say "Hello, World"'
Hello, World

So let's randomly sleep 100 times:

$ time perl6 -e 'for 1 .. 100 { sleep rand }'

real    0m50.567s
user    0m0.288s
sys 0m0.040s

OK, that took almost a minute. Now let's run that in parallel:

$ time perl6 -e 'await do for 1 .. 100 { start { sleep rand } }'

real    0m3.635s
user    0m0.367s
sys 0m0.100s

50 seconds down to 3 seconds. Not bad!

What does that code mean, though?

start means "start this code in another process and return a promise". A promise is merely a piece of asynchronous work that the system will try to complete. In other words, start returns a promise to complete this code in another process.

await takes a list of promises and waits for them to finish. It's a blocking function. Promises don't block, but await does:

$ time perl6 -e 'for 1 .. 100 { start { sleep rand } }'

real    0m0.284s
user    0m0.237s
sys 0m0.076s

(But don't do that. Spawned threads will be taken down when the program exits, so make sure you wait for them to finish)

Jonathan Worthington has a brilliant talk about Perl 6 concurrency and here's the video for it:

Or you can read this blog post, if you prefer that.

This is important. Having somewhat easy-to-use, working parallel programming in a language will be a killer feature, regardless of whether or not it's a dynamic language. That being said, you may not care about this. You want to know if the language is fun and easy.

There's a Learn Y in X Minutes post for Perl 6 which explains many of the concepts. That should get you up to speed on the basics rather quickly.

Yes, Perl 6 still has bugs. I found two rather quickly, but with rakudobrew it will be easier to stay up to date and the concurrency work is very interesting.

If you need to dive into the language, you can also read the Perl 6 docs. There's a lot of new terminology, but the language is now stable enough that a tutorial is merited.

If I had the opportunity, I think I'd spend a lot of my time porting a Web framework to Perl 6. Sadly, I have to pay the bills and I can't, but I'm finding Perl 6 to be a lovely thing to play with. How many of you can say you still love someone/something after almost a decade and a half (ooh, I'm going to hell for that last comment).

18 Comments

Lately you make a lot of comments you'll go to hell for. :-)

Something I did not properly appreciate until I started writing Erlang all the time was that Amdahl's law dominates highly concurrent code.

I really like the notion of Channels that Perl6 seems to have implemented in the spirit of the golang channels. They are brilliant.

Jonathon gave a surprise talk at MojoConf, the topic of which was of course the benefits that p6 would have for a non-blocking web framework like Mojolicious and I have to admit, it was the first time that I got a yearning for it. I came to Perl around 2008 and to me p6 had always been an impediment to p5. Finally I got excited. Now I really hope that perhaps Christmas will finally come.

That's funny, because, as it happens, Dancer was ported to Perl 6 a long while ago. :)

Thanks for the encouraging blog! Lifted my spirits.

Welll, "ported" is a bit of an overstatement :) But there is indeed a Dancer lookalike in Perl 6 for a while now: https://github.com/tadzik/bailador

rakudobrew should not blindly assume ~/.rakudobrew

due to disk space I installed it somewhere else, and the build process littered $HOME

$ perl6 -e 'say "Hello, World"' Hello, World

In some examples, it's actually an order of magnitude faster than Perl 5. That's astonishing given that Perl 5 is already the fastest dynamic language out there. In other examples, it's more than an order of magnitude slower

I appreciate that this wasn't the main point of your post, but: Perl 5 is "the fastest dynamic language"? That's quite a surprising assertion - what's it based on?

For some tasks, JS is up to 30x faster:

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=perl&lang2=v8&data=u32

This means an order of magnitude still isn't all that impressive.

Even Lua beats perl5:

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=perl&lang2=lua&data=u32

Perl is convenient, rather than fast. If you're lucky enough to have a task involving the right set of opcodes then sure, there's some great code in the perl core (and various XS modules) that'll give amazing results. But if you take into account object construction, method calls, 'my $self = shift', refcounting... the overhead adds up quickly: in my experience a typical application tends to be slower than the JS equivalent. Perl only starts to gain an edge if you compare things like _.map vs. perl's built-in map, which isn't really fair.

That's quite a surprising assertion – what's it based on?

It used to be true some while ago – and some of the basis for that still applies: Perl is still faster than CPython and PHP. It beat the pants off the old Ruby runtime but I have no idea how the new one stacks up. It also used to be faster than Javascript before that one hitched a ride on the JIT rocket. Lua was entirely off the map at the time.

So it’s still more or less “the fastest of the P languages”, at least for now – but no, it has been a while since it was last king of the entire hill.

It's worth noting that in jnthn's talk for YAPC::EU 2013, he uses the "async" keyword which is now written as "start" instead.

They're finally working on making Perl 6 fast. In some examples, it's actually an order of magnitude faster than Perl 5

To be fair, I think that's only the case for micro-benchmarks that compare Perl 6 built-in features with the equivalent features provided by bloated/unoptimized/neglected Perl 5 modules. For example, Perl 6's built-in Rat type vs Perl 5's Math::BigRat.

In macro-benchmarks that use many different language features to achieve something resembling a real-world task, Rakudo Perl 6 still tends to be very slow. Also, start-up time is an issue for command-line scripts (although if you're using Perl 5 + Moose, you'll already be used to that...)

Of course, the performance progress that the Rakudo developers have made over the course of the last 6 months is impressive, and if I understand correctly, the optimization work is far from finished. Like Ovid, I'm hopeful that Winter ahem Christmas is coming... :)

I'm quite excited to use this! I wonder if it will be possible to patch rakudobrew to work on Windows.

To be fair, I think that's only the case for micro-benchmarks that compare Perl 6 **built-in** features with the equivalent features provided by bloated/unoptimized/neglected Perl 5 **modules**. For example, Perl 6's built-in Rat type vs Perl 5's Math::BigRat.

No, actually, that is no longer true. There are cases where Rakudo/MoarVM keeps up with Perl 5 at built-ins, or beats it handily. At built-ins. Check out the benchmarks and see for yourself. There is a little “[Code]” button next to each benchmark that displays what is being measured; look for the graphs where the Rakudo/MoarVM line overlaps or soars above the Perl5 line.

(Select ones of these benchmarks – without the code – are included in jnthn’s YAPC::EU talk slides.)

The bigger ding here is that these are rather specific micro-benchmarks, and the overall practical performance does still trail Perl 5 by a wide margin.

But do you expect it to stay like this?

(Obviously trend lines ought not be extrapolated linearly, but I invite you to consider how long Java was the butt of many a joke about performance, how its VM is regarded nowadays, how long that transformation took, how much manpower it required, and to then contrast how long it took how much manpower for MoarVM to make the strides shown here.)

There are cases where Rakudo/MoarVM keeps up with Perl 5 at built-ins, or beats it handily. At built-ins. Check out the benchmarks and see for yourself.

I only see 3 kinds of micro-benchmarks on that page where Perl 6 comes in faster or as fast as Perl 5:

  1. Empty loops combined with integer arithmetic: while_empty_native, postwhile_nil_native, loop_empty_native

    This seems like an altogether archaic use-case from a P5 point of view (that is, probably not very meaningful with regards to real-life code).

  2. Rational number arithmetic (P6 built-in "Rat" vs P5 "use Math::BigRat"): rat_mul_div_cancel rat_harmonic

    This is what I was referring to in my previous comment.

  3. trim_string

    This one is interesting; although I'm not entirely convinced that
    my ($result) = $s =~ /^\s*(.*?)\s*$/s #P5
    is the best way to fairly translate
    $s.trim # P6

    For one thing, an assignment is happening in the P5 version whereas the P6 one seemingly just discards the result; also, the P5 version could be rewritten using s///r in a way that does not involve constructing a capture group.

As for the macro-benchmarks, P6 is considerably slower for all of them -- except for rc-mandelbrot, which uses complex number arithmetic (P6 built-in "Complex" vs P5 "use Math::Complex;") - again, the kind of thing I was referring to in my last comment.

But do you expect it to stay like this?

No, I hope Rakudo will get similarly fast as perl. But lets not call it before it happens... :)

But lets not call it before it happens... :)

The way I understand these results is “here is proof positive that it can”: a plausible promise. There are still seasons’ worth of low-hanging fruit ahead, anywhere you look, so the fact that we have these results now is an encouraging indicator of how high the effort might be able to reach.

Before this point, that was always the claim and the idea, and for some people it was the reasonable expectation, but there was no tangible evidence. It remains to be seen if the promise can be fulfilled, but these numbers do mark a new stage.

(FWIW, the results on integer arithmetic are interesting because of low-hanging fruit: many opportunities are still open for the runtime to specialise more generic numeric operations to integer, speeding up math across the board. Also, MoarVM is also already looking good in the concat-based benchmarks – an indication of the attainable string op performance. I’m looking forward to the future.)

If you are on Windows then the latest Rakudo* 2014-08 is available with a windows friendly .msi installer.

http://rakudo.org/downloads/star/rakudo-star-2014.08-moar.msi

rakudo works on windows, we also have an msi installer: http://rakudo.org/downloads/star/rakudo-star-2014.08-moar.msi

About Ovid

user-pic Freelance Perl/Testing/Agile consultant and trainer. See http://www.allaroundtheworld.fr/ for our services. If you have a problem with Perl, we will solve it for you. And don't forget to buy my book! http://www.amazon.com/Beginning-Perl-Curtis-Poe/dp/1118013840/