Benchmarking Rakudo releases. Is Raku Still slow?
Benchmarking Rakudo releases. Is Raku Still slow?
Context
Around the first 'public' release of Perl 6 (The x-mas release) I wrote a module that uses the libgumbo from google to parse html5 webpages.
It was faster and more robust than the existing HTML::Parser
module written in pure Perl 6.
To be fair to the module, the full html5 norm is rather lenghtly to implement.
I was using this to parse my list of favorite fan fiction on fimfiction.net
and try to make some stats around them.
It was still not super fast, like 0.5+ sec to parse one page (I have like 20+ pages of favorites). So each run of the script was rather slow.
At the time Perl 6 was still in stabilization phase and performance improvements were not really important. But after a while, it started to creep its ways out into Moar and Rakudo. So I wanted to see the improvement over the Rakudo release for my Gumbo module.
Building all Rakudo release
I already have done it, but I give another shot recently. I run a script that gets all rakudo releases listed in a text file, then compile it.
Unsurprisly it was not straightforward to build all releases from 2016.01
Perl change
Perl 5 chose to remove . in @INC
by default since 2016 so the Configure.pl
script that build Rakudo was not working properly. I had to fix
the script that build all rakudo release to add the current directory in PERL5LIB
.
Compile error
Release between 2018.05 and 2019 does not compile. Complaining about a symbol redefinition when linking Moar. I have really look at what is the cause so these releases get skipped.
Use the source, or not
Another small issue that prevented building release after 2018-2019 was I was downloading an archive of the source and not the release.
Since some submodules get added to rakudo/rakudo
some files were missing with just the source preventing Configure.pl
from working.
Benchmarking
Now that I have lots of rakudo releases it's time to start my benchmark script. It's a small script that starts 100 runs of another script that
outputs the time it took to convert the C struct from gumbo to XML
raku objects.
Here the raw result in seconds.
rakudo-2016.01.1, gumboxml.p6, 0.5593822334
rakudo-2016.02, gumboxml.p6, 0.51675639793
rakudo-2016.03, gumboxml.p6, 0.4856681754
rakudo-2016.04, gumboxml.p6, 0.46893925398
rakudo-2016.05, gumboxml.p6, 0.37183122053
rakudo-2016.06, gumboxml.p6, 0.29495558872
rakudo-2016.07.1, gumboxml.p6, 0.2703936729
rakudo-2016.08.1, gumboxml.p6, 0.2834315306
rakudo-2016.09, gumboxml.p6, 0.22285279441
rakudo-2016.10, gumboxml.p6, 0.2468772645
rakudo-2016.11, gumboxml.p6, 0.23989140535
rakudo-2016.12, gumboxml.p6, 0.23177488273
rakudo-2017.01, gumboxml.p6, 0.20607068872
rakudo-2017.02, gumboxml.p6, 0.22003044103
rakudo-2017.03, gumboxml.p6, 0.2253676812
rakudo-2017.04.3, gumboxml.p6, 0.21643373279
rakudo-2017.05, gumboxml.p6, 0.22202899438
rakudo-2017.06, gumboxml.p6, 0.22720643506
rakudo-2017.07, gumboxml.p6, 0.2313510098
rakudo-2017.08, gumboxml.p6, 0.18363233254
rakudo-2017.09, gumboxml.p6, 0.17739327075
rakudo-2017.10, gumboxml.p6, 0.16277988213
rakudo-2017.11, gumboxml.p6, 0.16266915587
rakudo-2017.12, gumboxml.p6, 0.1564215549
rakudo-2018.01, gumboxml.p6, 0.15112241018
rakudo-2018.02.1, gumboxml.p6, 0.15539236832
rakudo-2018.03, gumboxml.p6, 0.15672034753
rakudo-2018.04.1, gumboxml.p6, 0.1496397763
rakudo-2020.02.1, gumboxml.p6, 0.13660335827
rakudo-2020.05.1, gumboxml.p6, 0.13341074616
rakudo-2020.06, gumboxml.p6, 0.13528441194
rakudo-2020.07, gumboxml.p6, 0.13416307053
rakudo-2020.08.2, gumboxml.p6, 0.13747456749
rakudo-2020.09, gumboxml.p6, 0.13343004262
rakudo-2020.10, gumboxml.p6, 0.13358617365
rakudo-2020.11, gumboxml.p6, 0.15460961107
rakudo-2020.12, gumboxml.p6, 0.1448251052
rakudo-2021.02.1, gumboxml.p6, 0.14904339715
rakudo-2021.03, gumboxml.p6, 0.15159872921
rakudo-2021.04, gumboxml.p6, 0.1457165967
rakudo-2021.05, gumboxml.p6, 0.15416538145
rakudo-2021.06, gumboxml.p6, 0.15800617093
rakudo-2021.07, gumboxml.p6, 0.16087536077
rakudo-2021.08, gumboxml.p6, 0.16315127708
rakudo-2021.09, gumboxml.p6, 0.16488186848
rakudo-2021.10, gumboxml.p6, 0.27418679247
rakudo-2021.12, gumboxml.p6, 0.28623759495
rakudo-2022.02, gumboxml.p6, 0.24700117437
rakudo-2022.03, gumboxml.p6, 0.25029093914
rakudo-2022.06, gumboxml.p6, 0.23905013536
rakudo-2022.07, gumboxml.p6, 0.24230226215
rakudo-2022.12, gumboxml.p6, 0.23526197301
We can see that after 2016 performance improve up to 2020.10 then we lost performance, with a huge decrease in performance from 2021.09 et 2021.10.