Renormalist

This year at the Perl QA Hackathon I had three topics: benchmark update Perl until 5.24, enable CPAN test reporting on L4Linux, release a Net::SSH::Perl v2 to CPAN.

Part 1 - Benchmark::Perl::Formance

To keep a benchmark stable but still allow further development, last year I started to create separate bundles of existing benchmarks, starting with the "PerlStone2015" suite. Once settled I would only touch it for maintenance, and for newer developments I can fork it into an independent "PerlStone2016" suite where I could adapt the timings for newer hardware, other benchmarks, or particular language features.

This hackathon I reviewed and polished it to take reasonable runtime in "normal" mode so it does not take weeks to execute and also in "fastmode" where a benchmark produces results within 1-2 seconds.

Once I had these carefully polished I sync'ed my CPAN mirror and updated all of my multi-hundred Perl installations with the current CPAN dependencies and started benchmarking them in "fastmode" to get a tendency by end of the event. They ran repeatedly so I have 10+ data points per metric and Perl version for some statistical significance.

The dashboard is here.

First conclusions for the 5.23-5.24 era:

• improvements for small blocks entry/exit are clearly visible.
• algorithmic benchmarks like binarytrees, fannkuch, fasta, nbody, mandelbrot became remarkably faster
• others like regexdna and spectralnorm became already faster with 5.20, and could keep that speed
• some regex micro benchmarks run faster
• however, the regex engine micro benchmarks generally became slower since 5.18, but at least kept stable at that level

Please note, I could only run the scaled-down "fastmode" benchmarks. I will run the heavier-weight benchmarks throughout the next weeks.

Thanks to our sponsors

Thank you very much, FastMail, ZipRecruiter, ActiveState, OpusVL, Strato, SureVoiP, CV Library, Infinity Interactive, Perl Careers, MongoDB, think project!, DreamHost, Campus Explorer, and the Perl 6 community.

Sponsoring is always a difficult topic inside a company, I know it from both sides of the "money transaction", therefore I am very grateful for the sponsors' leap of faith into the event, and I can at the same time confirm that it is absolutely worth it. 30+ people working for a week in the same room for 16+ hours a day with zero communication barriers gives an incredible boost on bringing things forward.