I was fortunate to be invited to participate in the 2017 Perl 5 Core Hackathon (P5H) held in Amsterdam from Thursday, October 12 to Sunday, October 15. This event brings together contributors to the Perl 5 core development process -- often referred to as the "Perl 5 Porters" (P5P) -- for discussion on the current state and future of the language and for hacking to get us to that future state.

I don't consider myself a "perlguts" hacker, so quite a few of the discussions at P5H were above my head. I hope that other P5H participants will blog about those discussions. On a week-to-week basis my own contributions to P5P tend to fall into what broadly can be characterized as "QA for the core distribution": conducting and evaluating smoke test reports; responding to bug reports at https://rt.perl.org; and so forth. So, as the hackathon's opening approached, I prepared several mini-essays to serve as the basis for project discussions. I then led some of the discussions on those topics on the third and fourth days of the event. Here I'll describe some of the highlights of those discussions.

What Do We Want and Need from Smoke Testing?

Wiki reference

I initially raised this topic in my presentation at The Perl Conference: North America in Alexandria, Virginia, in June of this year (video). At the hackathon I noted that in the four months subsequent we had had a considerable increase in the volume of smoke test reports being received (see Tony Cook's summaries), in particular from new contributor Carlos Guevara. But we still lack smoke-testing of branches other than blead; little capacity to follow up on reports of failures on platforms other than Linux; and over-reliance on individual volunteers. (Slide presentation)

We agreed that we need a Jenkins-like testing rig which functions as an early warning system for build or test failures which a committer failed to detect before pushing. This rig should be very simple -- probably just Linux non-threaded and threaded builds. We will first explore a Travis configuration which performs these build and reports failures to perl5-porters mailing list.

We will also investigate whether Travis configurations can be used to automate smoke-testing of branches other than blead.

We also decided to explore additional ways to obtain "Blead Breaks CPAN" failure data and respond thereto. One approach would be to explore the data which CPANtesters.org collects on tests of CPAN libraries against the monthly development releases of Perl and leverage our efforts upon that data. Since P5H we have learned that a number of individuals, not previously know to us, have created their own rigs for this purpose. We will try to build on those efforts as well.

Do We Need a Tertiary Test Suite?

Wiki reference

This P5H topic also came out of my TPC::NA::2017 presentation. A tertiary test suite would test against blead (or, more likely, monthly development releases) those applications which are:

• written in Perl; and
• widely distributed as packages in the major Linux and BSD distributions.

We would run this test suite periodically over the course of a year-long development cycle, but particularly when we propose a deprecation or subsequent fatalization of some Perl 5 functionality. This test suite would be used as the basis for dialog with the "Perl maintainers" of those open-source distributions and guide our further actions.

At P5H we felt that this would be approached best not by trying to build a comprehensive solution, but rather by conducting a modest pilot project. A task force of four P5H participants -- including two Perl maintainers from the Debian Linux community -- will identify some programs in Debian which are dependent on Perl and will run those programs' individual test suites against Perl's monthly development releases.

Who Made This Possible?

Sponsors for the Perl 5 Hackathon 2017

This conference could not have happened without the generous contributions from the following companies:

• Booking.com
• cPanel
• craigslist
• bluehost
• Assurant
• Grant Street Group

And on a More Personal Note

In addition, I benefited from the generous hospitality of some long-time Dutch Perlmongers in the days leading up to the event:

• Mark, Cleo and Tycho Overmeer
• Wendy van Dijk and Liz Mattijsen

Also thanks to the Bruxelles.pm for a pre-hackathon social meeting.