Kenichi Ishigaki

Last year at the Perl Toolchain Summit (PTS) in Lyon, I left three draft pull requests: one about the class declaration introduced in Perl 5.37, one about the PAUSE on docker, and one about multifactor authentication. I wanted to brush them up and ask Andreas König to merge some, but which should I prioritize this year?

I focused on the web UI in the past because other people tended to deal with the PAUSE backend, especially its indexer. But this year, when I was able to start thinking about my plan, Ricardo Signes and Matthew Horsfall had already expressed their plan about migrating the PAUSE to a new server. I was unsure if they would use my docker stuff, but I could safely guess I didn't need to touch it. I also thought we wouldn't have time to finish the multifactor authentication because it would need to change the PAUSE itself and the uploader clients, and Ricardo maintained the most favorite uploader module. The change for the new class detection was simple, but that didn't mean the result would also be predictable. I decided to investigate how the 02packages index would change first.

I needed to find a way to rebuild the index from scratch to see the differences. I wrote a script to gather author information from a CPAN mirror and filled the PAUSE's user-related tables with dummy data. I wrote another script to register my distributions in the mirror to my local PAUSE. The PAUSE would complain if I registered an older distribution after a newer one, so I had to gather all the information about my distributions and sort them by creation time. It seemed fine now, but it soon started hanging up when I increased the number of the distributions to register. The PAUSE daemon spawned too many child indexer processes and ate up all the memory I allocated to a virtual machine. After several trials and errors, I limited the number of child processes with Parallel::Runner, which I used for the CPANTS for years. Even if it weren't acceptable to Andreas for some reason, it should be easy to ask for the author's help because he (Chad Granum) would be at the PTS. I also had to fix a deadlock in the database due to the lack of proper indices. Matthew had already made a pull request last year, but I misread it and fixed the issue in a different (and inefficient) way.

Now that the script ran without hanging, I compared the generated 02packages index with the one in the mirror. I found more than four thousand lines of difference. I modified my local PAUSE clone to see why that happened. It looked like most of them were removed due to historical changes in the indexing policy, but instead of digging into it further, I decided to use what I got as a reference point and started changing the indexer. After several comparisons, I modified my local indexer to take care of the byte order mark and let it look for class declarations only when a few "use" statements were found. I applied the same changes to my Parse::PMFile module and made two releases before the PTS.

Day 1 of the PTS in Lisbon started with a discussion of the PAUSE migration. While the migration team was preparing, I asked Andreas to merge some of the existing small pull requests. The first one was to replace Travis CI with GitHub Actions by Ricardo. Unfortunately, it turned out that Test::mysqld and App::yath didn't work well in the GitHub Actions environment. I asked Chad for advice, but we couldn't make it work, so I tweaked the workflow file to use the good old "prove" command. The second was to improve password generation using Crypt::URandom by Leon Timmermans. I made another pull request to add it to the cpanfile for GitHub Actions. It might be better to modify our Makefile.PL to use ExtUtils::MakeMaker::CPANfile so that we wouldn't need to maintain both cpanfile and Makefile.PL. Maybe next time.

After dealing with a few more issues and pull requests, we moved on to class detection. As a starter, I asked Andreas to merge a years-old pull request by Ricardo to make the package detection stricter and then a pull request about the BOM I made. We discussed whether we could ignore class declarations by older modules such as MooseX::Declare. With Andreas' nod, I made another pull request and asked Ricardo and Matthew to review it.

I started day two by adding tests about the class detection with Module::Faker. I made another pull request to create a new 08pumpking index per Graham Knop's request, which MetaCPAN would eventually use. After merging them and a few more pull requests, I recreated a draft pull request on the multifactor authentication with pieces I couldn't implement last year (such as recovery codes). We also discussed the deadlock issue. In the end Andreas chose my pull request plus a commit from the one by Matthew. I was sorry we encountered a disk shortage while adding indices. Robert Spier helped us and optimized the database. By the end of the day, we had a few more pull requests merged, including the one for Parallel::Runner, with the help of Chad.

Day 3 was Deployment day. The migration team was busy, and there was no room for other stuff. I walked through the open issues, replied to some, and made a few small pull requests, hoping to revisit them in the future.

On day 4, I spent some time trying to figure out why uploading a large file to the new server didn't work, but in vain. I also attended a discussion about future PAUSE development. It would be nice to see the development continue after the offline event.

Many thanks to Breno Oliveira, Philippe Bruhat, and Laurent Boivin for organizing this event again and to our generous sponsors.

Monetary sponsors: Booking.com, The Perl and Raku Foundation, Deriv, cPanel, Inc Japan Perl Association, Perl-Services, Simplelists Ltd, Ctrl O Ltd, Findus Internet-OPAC, Harald Joerg, Steven Schubiger.

In-kind sponsors: Fastmail, Grant Street Group, Deft, Procura, Healex GmbH, SUSE, Zoopla.