Perl Toolchain Summit 2025 in Leipzig

In the past, it took two years to merge my first PAUSE on Plack branch into the master and three years to merge the next PAUSE on Mojolicious (actually, two years to deploy and another year to merge). Now the question was: how long would it take to merge the next big thing, multifactor authentication for PAUSE? Two years, three years, or maybe four years this time? I already had a two-year-old draft branch and initially wished to merge it this year. However, things went differently.

The first MFA branch requested that you send your authentication code only when posting essential data, such as uploading a distribution or editing your account. If you choose to enable MFA, you may need to update your tools to allow you to send an extra code; otherwise, you shouldn't have to take any action. I knew it was suboptimal. It would only protect people keen on security. However, it would be less likely to hinder people from uploading a new distribution. Considering that we may soon have to ask people to include SBOM metadata in their distributions, keeping it easy to upload could have a more positive value.

That said, the branch wouldn't help if we had to force all users to enable MFA at login time for some reason. I wanted to laugh the idea away as unfounded fear, but who knows? I wouldn't be able to implement such a significant change during a four-day PTS, and I couldn't tell when the day would come. So, I started a new experiment to get proof of the concept beforehand.

The only way I could think of at the time was to add a new PAUSE application that allows you to log in to PAUSE by posting your credentials and then an additional MFA code via web forms (instead of the current basic authentication mechanism). If I mount it on a different path from the current PAUSE, you can choose which one to use, and it would be easy for us to drop one when necessary. However, it also means I must update two places whenever I make a change. I couldn't tell if it would be worth the cost, but one thing was sure. If we decide to mount a new application on a different path, existing tools, such as uploaders and deleters, will not work with the new app. And if we didn't need to keep compatibility with those tools, we would be able to change the app more boldly: we could use path information to determine a route (instead of an ACTION query parameter); we could drop pause99_ prefix all the form fields currently had; we could even go further to implement the design idea Babs Veloso gave us during the PTS 2018 in Oslo (You might remember my presentation at PerlCon 2019 in Riga).

I didn't have time to implement everything. I barely managed to make the new app work. It had almost no tests, and its session data was still in a signed cookie, not in the database. I asked Ricardo Signes and Matthew Horsfall for a review on the day I left home for the PTS, not knowing they wouldn't be attending this year.

I usually fly to the nearest airport to the venue, but it was too expensive to fly from Tokyo to Leipzig. To minimize the travel cost, I replaced the last leg with a train trip and left home earlier to avoid the Japanese holiday week. I stayed two nights in Frankfurt and arrived in Leipzig by train on the day before the PTS. However, I was still living as if I were in a different time zone. I couldn't stay awake and failed to attend the welcoming pre-dinner.

Day 1 started with a stand-up session. Missing a few people, I announced I had a new MFA branch. After the stand-up, however, Andreas König expressed his discontent about MFA over his breakfast. The only option for me was to retreat for a while.

So, instead of improving the MFA branch, I began reviewing all the open issues and pull requests. I asked Andreas to merge some obvious requests. One of them was by Thibault Duponchelle. He showed me his version of PAUSE on Docker branch and asked me how to ignore a git issue. I remembered I had a workaround, but it was only on my local branch (I made it a pull request on day 3). I also discussed a Local namespace issue with him.

Meanwhile, some of the CPAN security team came to us to see how the new MFA branch worked. I explained the current status and spent the rest of the day converting tests for the new app.

On day 2, I updated Mojolicious::Plugin::WithCSRFProtection and asked Andreas to merge a few more easy pull requests, including the one about the Local namespace. I made several more requests, but to my regret, I had to ask Andreas to revert some of them after the PTS. We also discussed an email address issue Andreas put the highest priority on this year. There was already a pull request by Ricardo, but it was part of a significant refactoring, and we didn't want to merge it when he was absent. Instead, I suggested a minimum fix to omit the display name part, which may be less polite but should cause fewer problems.

In addition, I enjoyed a presentation on Perl 5.42 by Paul Evans. I also joined a discussion about "abandoned" CPAN distributions, but this one didn't go well. Learning the number of inactivated accounts also depressed me. I attended a guided city tour for fresh air. Daniel Böhmer, the local organizer, showed us some interesting spots that I wouldn't visit without him. Vielen Dank.

At the social dinner after the tour, I asked Timothy Legge (of the security team) a lot of questions about MFA. Thanks to his kind answers, we got a clearer picture of the issue.

Although I felt the atmosphere had changed, I continued to fix minor issues on day 3. As only two members of the PAUSE team presented this year, I also reviewed the issues and requests of other team members and asked Andreas to merge a few, with or without slight modifications.

I also had a conversation about the EU Cyber Resilience Act with Salve Nilsen. Breno Oliveira gave me some ideas for the MFA branch as well.

Now that I had fixed what I thought I could do this year, I applied the same changes I made this year to the MFA branch on day 4. I also discussed what I should do the next time with Andreas and some of the security team. Stig Palmquist provided us with several ideas and suggested that we should let users use a longer password before proceeding with anything else. We also discussed other security issues. Some of us favored the idea of pinning down all the CPAN modules PAUSE uses (by Carton or Carmel), but Andreas rejected it. I made a pull request to update the security database entry on CPAN::Checksums while trying Test::CVE.

Many thanks to Daniel Böhmer, Tina Müller, Breno Oliveira, Philippe Bruhat, and Laurent Boivin for organizing this event again and to our generous sponsors:

Monetary Sponsors

Booking.com, WebPros, CosmoShop, Datensegler, OpenCage, SUSE, Simplelists Ltd, Ctrl O Ltd, Findus Internet-OPAC, plusW GmbH

In-kind sponsors

Grant Street Group, Fastmail, shift2, Oleeo, Ferenc Erki

Community Sponsors

The Perl and Raku Foundation, Japan Perl Association, Harald Joerg, Alexandros Karelas (PerlModules.net), Matthew Persico, Michele Beltrame (Sigmafin), Rob Hall, Joel Roth, Richard Leach, Jonathan Kean, Richard Loveland, Bojan Ramsa.

Perl Toolchain Summit 2024 in Lisbon

Last year at the Perl Toolchain Summit (PTS) in Lyon, I left three draft pull requests: one about the class declaration introduced in Perl 5.37, one about the PAUSE on docker, and one about multifactor authentication. I wanted to brush them up and ask Andreas König to merge some, but which should I prioritize this year?

I focused on the web UI in the past because other people tended to deal with the PAUSE backend, especially its indexer. But this year, when I was able to start thinking about my plan, Ricardo Signes and Matthew Horsfall had already expressed their plan about migrating the PAUSE to a new server. I was unsure if they would use my docker stuff, but I could safely guess I didn't need to touch it. I also thought we wouldn't have time to finish the multifactor authentication because it would need to change the PAUSE itself and the uploader clients, and Ricardo maintained the most favorite uploader module. The change for the new class detection was simple, but that didn't mean the result would also be predictable. I decided to investigate how the 02packages index would change first.

I needed to find a way to rebuild the index from scratch to see the differences. I wrote a script to gather author information from a CPAN mirror and filled the PAUSE's user-related tables with dummy data. I wrote another script to register my distributions in the mirror to my local PAUSE. The PAUSE would complain if I registered an older distribution after a newer one, so I had to gather all the information about my distributions and sort them by creation time. It seemed fine now, but it soon started hanging up when I increased the number of the distributions to register. The PAUSE daemon spawned too many child indexer processes and ate up all the memory I allocated to a virtual machine. After several trials and errors, I limited the number of child processes with Parallel::Runner, which I used for the CPANTS for years. Even if it weren't acceptable to Andreas for some reason, it should be easy to ask for the author's help because he (Chad Granum) would be at the PTS. I also had to fix a deadlock in the database due to the lack of proper indices. Matthew had already made a pull request last year, but I misread it and fixed the issue in a different (and inefficient) way.

Now that the script ran without hanging, I compared the generated 02packages index with the one in the mirror. I found more than four thousand lines of difference. I modified my local PAUSE clone to see why that happened. It looked like most of them were removed due to historical changes in the indexing policy, but instead of digging into it further, I decided to use what I got as a reference point and started changing the indexer. After several comparisons, I modified my local indexer to take care of the byte order mark and let it look for class declarations only when a few "use" statements were found. I applied the same changes to my Parse::PMFile module and made two releases before the PTS.

Day 1 of the PTS in Lisbon started with a discussion of the PAUSE migration. While the migration team was preparing, I asked Andreas to merge some of the existing small pull requests. The first one was to replace Travis CI with GitHub Actions by Ricardo. Unfortunately, it turned out that Test::mysqld and App::yath didn't work well in the GitHub Actions environment. I asked Chad for advice, but we couldn't make it work, so I tweaked the workflow file to use the good old "prove" command. The second was to improve password generation using Crypt::URandom by Leon Timmermans. I made another pull request to add it to the cpanfile for GitHub Actions. It might be better to modify our Makefile.PL to use ExtUtils::MakeMaker::CPANfile so that we wouldn't need to maintain both cpanfile and Makefile.PL. Maybe next time.

After dealing with a few more issues and pull requests, we moved on to class detection. As a starter, I asked Andreas to merge a years-old pull request by Ricardo to make the package detection stricter and then a pull request about the BOM I made. We discussed whether we could ignore class declarations by older modules such as MooseX::Declare. With Andreas' nod, I made another pull request and asked Ricardo and Matthew to review it.

I started day two by adding tests about the class detection with Module::Faker. I made another pull request to create a new 08pumpking index per Graham Knop's request, which MetaCPAN would eventually use. After merging them and a few more pull requests, I recreated a draft pull request on the multifactor authentication with pieces I couldn't implement last year (such as recovery codes). We also discussed the deadlock issue. In the end Andreas chose my pull request plus a commit from the one by Matthew. I was sorry we encountered a disk shortage while adding indices. Robert Spier helped us and optimized the database. By the end of the day, we had a few more pull requests merged, including the one for Parallel::Runner, with the help of Chad.

Day 3 was Deployment day. The migration team was busy, and there was no room for other stuff. I walked through the open issues, replied to some, and made a few small pull requests, hoping to revisit them in the future.

On day 4, I spent some time trying to figure out why uploading a large file to the new server didn't work, but in vain. I also attended a discussion about future PAUSE development. It would be nice to see the development continue after the offline event.

Many thanks to Breno Oliveira, Philippe Bruhat, and Laurent Boivin for organizing this event again and to our generous sponsors.

Monetary sponsors: Booking.com, The Perl and Raku Foundation, Deriv, cPanel, Inc Japan Perl Association, Perl-Services, Simplelists Ltd, Ctrl O Ltd, Findus Internet-OPAC, Harald Joerg, Steven Schubiger.

In-kind sponsors: Fastmail, Grant Street Group, Deft, Procura, Healex GmbH, SUSE, Zoopla.

Next stable DBD::SQLite will be released in the middle of September

DBD::SQLite 1.73_01 (with SQLite 3.42.0) is a release candidate for the next stable DBD::SQLite. This release is mainly to upgrade the bundled SQLite library.

I'll wait for about a month and release 1.74 in the middel of September if there's no blocker nor request to wait for more. Thank you for your patience.

Perl Toolchain Summit 2023 in Lyon

Four years have passed since the last Perl Toolchain Summit (PTS) in Marlow. I planned to continue working on PAUSE's web UI, but I didn't exactly remember what to do. So the first thing I did at home before the PTS was to read through the PAUSE issues and do some triage. I also resumed a virtual machine that held PAUSE clones I had worked on. There I found an untracked docker-compose.yml. It was incomplete. I must have given it up because I already had a working environment. However, I remembered a few people wanted an easier way to install PAUSE. It would be helpful if they could run PAUSE…

Next stable DBD::SQLite will be released at the beginning of November

DBD::SQLite 1.71_07 (with SQLite 3.39.4) is a release candidate for the next stable DBD::SQLite. This release is mainly to address a security hole found in SQLite, plus a few performance issues for perl built with -DDEBUGGING. See Changes for other fixes and changes.

This time I'll wait for about a week and release 1.72 at the beginning of November if there's no blocker nor request to wait for more. Thank you for your patience.