Benchmarking Rakudo releases. Is Raku Still slow?
Benchmarking Rakudo releases. Is Raku Still slow?
Context
Around the first 'public' release of Perl 6 (The x-mas release) I wrote a module that uses the libgumbo from google to parse html5 webpages.
It was faster and more robust than the existing HTML::Parser module written in pure Perl 6.
To be fair to the module, the full html5 norm is rather lenghtly to implement.
I was using this to parse my list of favorite fan fiction on fimfiction.net and try to make some stats around them.
It was still not super fast, like 0.5+ sec to parse one page (I have like 20+ pages of favorites). So each run of the script was rather slow.
At the time Perl 6 was still in stabilization phase and performance improvements were not really important. But after a while, it started to creep its ways out into Moar and Rakudo. So I wanted to see the improvement over the Rakudo release for my Gumbo module.
Building all Rakudo release
I already have done it, but I give another shot recently. I run a script that gets all rakudo releases listed in a text file, then compile it.
Unsurprisly it was not straightforward to build all releases from 2016.01
Perl change
Perl 5 chose to remove . in @INC by default since 2016 so the Configure.pl script that build Rakudo was not working properly. I had to fix
the script that build all rakudo release to add the current directory in PERL5LIB.
Compile error
Release between 2018.05 and 2019 does not compile. Complaining about a symbol redefinition when linking Moar. I have really look at what is the cause so these releases get skipped.
Use the source, or not
Another small issue that prevented building release after 2018-2019 was I was downloading an archive of the source and not the release.
Since some submodules get added to rakudo/rakudo some files were missing with just the source preventing Configure.pl from working.
Benchmarking
Now that I have lots of rakudo releases it's time to start my benchmark script. It's a small script that starts 100 runs of another script that
outputs the time it took to convert the C struct from gumbo to XML raku objects.
Here the raw result in seconds.
rakudo-2016.01.1, gumboxml.p6, 0.5593822334
rakudo-2016.02, gumboxml.p6, 0.51675639793
rakudo-2016.03, gumboxml.p6, 0.4856681754
rakudo-2016.04, gumboxml.p6, 0.46893925398
rakudo-2016.05, gumboxml.p6, 0.37183122053
rakudo-2016.06, gumboxml.p6, 0.29495558872
rakudo-2016.07.1, gumboxml.p6, 0.2703936729
rakudo-2016.08.1, gumboxml.p6, 0.2834315306
rakudo-2016.09, gumboxml.p6, 0.22285279441
rakudo-2016.10, gumboxml.p6, 0.2468772645
rakudo-2016.11, gumboxml.p6, 0.23989140535
rakudo-2016.12, gumboxml.p6, 0.23177488273
rakudo-2017.01, gumboxml.p6, 0.20607068872
rakudo-2017.02, gumboxml.p6, 0.22003044103
rakudo-2017.03, gumboxml.p6, 0.2253676812
rakudo-2017.04.3, gumboxml.p6, 0.21643373279
rakudo-2017.05, gumboxml.p6, 0.22202899438
rakudo-2017.06, gumboxml.p6, 0.22720643506
rakudo-2017.07, gumboxml.p6, 0.2313510098
rakudo-2017.08, gumboxml.p6, 0.18363233254
rakudo-2017.09, gumboxml.p6, 0.17739327075
rakudo-2017.10, gumboxml.p6, 0.16277988213
rakudo-2017.11, gumboxml.p6, 0.16266915587
rakudo-2017.12, gumboxml.p6, 0.1564215549
rakudo-2018.01, gumboxml.p6, 0.15112241018
rakudo-2018.02.1, gumboxml.p6, 0.15539236832
rakudo-2018.03, gumboxml.p6, 0.15672034753
rakudo-2018.04.1, gumboxml.p6, 0.1496397763
rakudo-2020.02.1, gumboxml.p6, 0.13660335827
rakudo-2020.05.1, gumboxml.p6, 0.13341074616
rakudo-2020.06, gumboxml.p6, 0.13528441194
rakudo-2020.07, gumboxml.p6, 0.13416307053
rakudo-2020.08.2, gumboxml.p6, 0.13747456749
rakudo-2020.09, gumboxml.p6, 0.13343004262
rakudo-2020.10, gumboxml.p6, 0.13358617365
rakudo-2020.11, gumboxml.p6, 0.15460961107
rakudo-2020.12, gumboxml.p6, 0.1448251052
rakudo-2021.02.1, gumboxml.p6, 0.14904339715
rakudo-2021.03, gumboxml.p6, 0.15159872921
rakudo-2021.04, gumboxml.p6, 0.1457165967
rakudo-2021.05, gumboxml.p6, 0.15416538145
rakudo-2021.06, gumboxml.p6, 0.15800617093
rakudo-2021.07, gumboxml.p6, 0.16087536077
rakudo-2021.08, gumboxml.p6, 0.16315127708
rakudo-2021.09, gumboxml.p6, 0.16488186848
rakudo-2021.10, gumboxml.p6, 0.27418679247
rakudo-2021.12, gumboxml.p6, 0.28623759495
rakudo-2022.02, gumboxml.p6, 0.24700117437
rakudo-2022.03, gumboxml.p6, 0.25029093914
rakudo-2022.06, gumboxml.p6, 0.23905013536
rakudo-2022.07, gumboxml.p6, 0.24230226215
rakudo-2022.12, gumboxml.p6, 0.23526197301
We can see that after 2016 performance improve up to 2020.10 then we lost performance, with a huge decrease in performance from 2021.09 et 2021.10.
I blog about Perl 6.
This is great, I would suggest reposting at every pre-release...because that is the time one can scrutinise the changes ad address them to prevent continuous degradation. Features get added that inevitably impacts the performance. identification of performance bottlenecks are better real-time rather than retrospective.