This weekend I was once again privileged to attend the Perl Toolchain Summit (PTS). This year it was held in the lovely city of Leipzig.

The PTS continues to be my favourite technical event of the year. In part this is because I get to meet old friends and make new ones, but it's also because the summit really serves its purpose and I am able to make so much progress on the projects I have which belong in Perl's toolchain ecosystem.

PTS isn't a conference - it's a four-day working meeting. It brings together people working on toolchain projects to solve common problems and push the work forward. I did get a lot of work done, but that's not the main focus, for me anyway. I see it as a time to solve problems and plan the way forward, and for me PTS facilitates that in the most wonderful fashion.

Numerous times after hitting a problem or having some question I was able to walk a few feet to the person in the world best qualified to solve the problem or answer the question. And a number of folk also came to me with questions or reported problems so I hope I was able to help in a similar way.

So, in terms of results, I:

• Made three Devel::Cover releases
• Fixed Devel::Cover to work with 5.42.0 (to be)
• Discovered and reported a perl problem during Devel::Cover testing (missing ^^= operator)
• Got cpancover.com working on btrfs with compression to allow more coverage reports on the server.
• Merged in all outstanding Devel::Cover PRs
• Closed out a few tickets

But beyond that I was able to have chats with various people and groups regarding all sorts of areas touching on coverage, testing, hosting, the perl core, build systems and various other adjacent topics. And this was the real value of the event. I came with a list of things to work on, knowing I wouldn't be able to do all of them. And whilst I worked on several topics I had planned, and especially the most important ones, I ended up not touching others and moving some in other directions based on discussions with people who understand things better than I.

In particular I want to call out Ferenc Erki for suggesting that the solution to my problem of running out of disk space for cpancover reports was not anything I was considering, but rather migrating to a filesystem which implements transparent compression, and then helping me implement that. I'm still playing with the small details, but this should solve that particular problem for a good few years.

And to give just one example of the value in bringing folk together, I was looking through some recent Devel::Cover tickets and noticed one related to using Devel::Cover with Module::Build::Tiny. I had previously put that to one side because it didn't mean much to me. But Leon Timmermans was sitting almost next to me and so I showed him the bug report. He immediately identified the problem and showed me a ticket and PR he had created a while back to implement the solution, but he hadn't had a use case and so he wasn't sure whether or not to merge it. Well, now Devel::Cover provided the use case and so we have two previously ignored tickets now referencing each other and a solution ready to go.

Part of the reason that this year I worked on areas I hadn't initially expected was because we were missing a few folk who regularly attend PTS. For some this was due to other commitments. For some the reason was more sad.

But this allowed a number of folk to attend for the first time. And this not only enhanced the summit itself, but hopefully will be of benefit to the Perl toolchain in general over the years.

PTS would not be possible without a considerable amount of support from many people. Obviously this includes the folk who give up their time to attend. But I'd also like to recognise Salve, who kicked the whole thing off, the organisers of every QAH/PTS since then for keeping the torch burning and, if I may mix my metaphors even further, the organisers of this year's event for knocking the ball out of the park. Big thanks to

• Daniel Böhmer, local organiser and Leipzig tour guide extraordinaire
• Tina Müller, local(ish) organiser
• Philippe Bruhat, organiser
• Breno de Oliveira, organiser
• Laurent Boivin, finances

Monetary Sponsors

Booking.com, WebPros, CosmoShop, Datensegler, OpenCage, SUSE, Simplelists Ltd, Ctrl O Ltd, Findus Internet-OPAC, plusW GmbH

In-kind sponsors

Grant Street Group, Fastmail, shift2, Oleeo, Ferenc Erki

Community Sponsors

The Perl and Raku Foundation, Japan Perl Association, Harald Joerg, Alexandros Karelas PerlModules.net, Matthew Persico, Michele Beltrame Sigmafin, Rob Hall, Joel Roth, Richard Leach, Jonathan Kean, Richard Loveland, Bojan Ramsa

Sometimes life catches up with you. I've felt that way for the last few years and I'm probably not alone.

During that time the cpancover project has basically just been plodding along, pretty much just working. As new modules were uploaded to CPAN, cpancover would pick them up, calculate the test coverage, and make the results available to be displayed on metacpan, along with detailed output on cpancover.com.

A little while ago I decided it was probably about time that I should update the OS and perl version and libraries and stuff.

And it went terribly.

Last week was the Perl Toolchain Summit. This is the annual event (skipping 2020-2022 of course) where about thirty of the folk who build, maintain and extend much of the Perl infrastructure get together to work on things which really benefit from people being together in the same room, and from setting aside some time to work on these projects with minimal distractions. This overlaps to an extent with core perl work, but the perl core isn't the main focus of the summit. Somehow I've managed join in with most of the events and, for me, it's the most important Perl event on my calendar.

My primary goal at this year's summit was to update the infrastructure around cpancover. For a while now I've been working more and more closely with the metacpan team, initially on displaying coverage results on metacpan, and more recently on working to bring our environments closer together with the idea that cpancover can benefit from the excellent work the metacpan team are doing. This has become more important as I have shown by managed to break a part of my only environment for cpancover.

So, after arriving in Lisbon for this year's PTS, I wanted to spend a time working with the metacpan folk on deciding how to push things forward in this area, and on actually doing so. I had a number of discussions with Joel and with Leo and settled on the way forward. Joel and Leo spent a lot of time working on the metacpan infrastructure, and cpancover became almost like a first customer for the work they are doing, after metacpan itself of course.

But we also looked at how we could fix up cpancover specifically in the short term in a way which would make later migration as simple as possible.

So apart from making these decisions, I also spent time on starting to implement things. I made a couple of Devel::Cover releases during the summit, which were focussed on updating code for infrastructure, as well as incorporating some more modern practices and tooling, such as linters and formatters.

I also made sure Devel::Cover was tested against the latest perl development releases. With a major release just days away this was, I think, the first time for many years where I didn't know there were problems with Devel::Cover on the latest perl development release. But it was good to confirm this. This was also nice in that it left more time for working on other matters.

In cpancover, I have always generated coverage for individual modules in docker containers. This provides a level of control for misbehaving or malicious modules when the tests are run with Devel::Cover. The goal is now to run the main controlling process in a docker container too, to isolate it from the underlying OS. So this work is now started. Unfortunately I didn't manage to finish it, but I did remove all the problems which were stopping me even getting to the point where I could work on it, which feels like a huge success.

So now, I think, it's just a Simple Matter Of Programming to get this done. And then, when the metacpan infrastructure is available we should be able to make use of that to provide more timely results, redundancy, development environments which match what is happening in production, and all that good stuff.

One thing I didn't get around to was looking at some work that Nicholas Clark did and presented at the German Perl Workshop (https://act.yapc.eu/gpw2024/talk/7872) to handle cases where a test suite doesn't even load a module, and the uncovered lines in that module aren't reported. I hope to get to that soon.

But I did manage to spend time talking with many people about work they are doing, covering topics from Test2 to YAMLScript to handling ancient Perl codebases to replacing ppaddr in the perl optree.

One of the things which always feels great about PTS is when you find a bug in a module you're using, and you can chat to the person who wrote the module and they immediately fix the problem. Or vice versa. That happened a few times this year - and it's always nice to get immediate feedback.

And, as usual, I came away with a longer TODO list than I arrived with. But, crucially, I came away unblocked on key design decisions and all the annoying bits of work that just needed to be done before real progress could be made.

I'm really grateful to everyone involved to make this event happen. That includes the organisers, Laurent Boivin, Philippe Bruhat and Breno de Oliveira and likely others behind the scenes. Laurent and Philippe have long been involved in the process and have the knowledge and experience to make the event run smoothly. Breno organised the local details from across an ocean and made sure the event ran with nary a hitch.

I'm also thankful to everyone who attended, often using their own holiday time to do so, and who helped me directly, gave talks or presentations, and generally contributed to the productive environment.

And, of course, none of this would have happened without our sponsors. A big thanks to all of them for their donations, whether as cash or in kind.

Monetary sponsors: Booking.com, The Perl and Raku Foundation, Deriv, cPanel, Inc Japan Perl Association, Perl-Services, Simplelists Ltd, Ctrl O Ltd, Findus Internet-OPAC, Harald Joerg, Steven Schubiger.

In kind sponsors: Fastmail, Grant Street Group, Deft, Procura, Healex GmbH, SUSE, Zoopla.

After a break of four years, it has been my privilege to attend the 13th Perl Toolchain Summit (née Perl QA Hackathon). This is the third time the summit has been held in Lyon and the tenth summit I have been able to attend. PTS is a really important event in the Perl calendar where those working on the Perl toolchain and in Perl QA get to meet together for four days of discussions, decisions on the future of Perl, and hacking.

Arriving late on Wednesday evening, I had an overfull list of things I wanted to talk to people about and work I wanted to do. In the end I didn't even get half way though my list, but that was expected.

I spent much of Thursday just getting things up and running again. It turns out that spending four years barely keeping a project alive leads to a fair amount of bitrot. So I updated servers and software and keys and did all sort of unglamorous stuff that I had just been putting off.

One thing I had thought was urgent was finding new hosting for cpancover. So I discussed this with the nice metacpan folk. They're in the process of worrying about all this sort of stuff and, since the requirements for cpancover have turned out to be less urgent than they initially seemed, I'm just going to let the metacpan folk do their stuff. I'll prepare cpancover and then, hopefully, when we're all ready, just piggyback on their hard work.

Almost since the start, cpancover has generated coverage data in containers, for security, to constrain resources and for ease. In order to move to new infrastructure it will be easiest if the rest of cpancover is containerised too. I didn't get a chance to work on this during the summit, but this is now high on the priority list.

Devel::Cover has a very close relationship with the perl core. Most of the logic for collecting coverage data is written in XS (C code which calls perl internal functions). This makes it rather susceptible to changes in the perl core. Perl has an annual release schedule and the 5.38.0 release is imminent. Fortunately, PTS is scheduled for shortly before the release which gives me the opportunity to make any changes necessary in Devel::Cover to account for the updates in the new release. On Friday I made a release of Devel::Cover which passes all its tests and so will allow people to use it with 5.38.0. But some of the behaviour has changed due to changes in perl's optree. During the rest of the summit I was able to restore the full functionality and make another release which won't artificially reduce coverage numbers. In this I was greatly helped by other summit attendees who were able to point me in the right direction or just directly tell me the changes I needed to make based on their knowledge of the core.

This is one of the major benefits of PTS. When there are problems to be solved, people who can help are often sitting just a few feet away and are ready to assist.

One of the discussions held during PTS was about how far back the toolchain and other tools should continue working. Based on this, and the current minimum versions of Devel::Cover dependencies, I raised the minimum perl version supported by Devel::Cover from 5.10 to 5.12. The toolchain is currently targeting 5.16 but I don't need to move there yet.

During the summit I also merged in all the outstanding PRs, and worked on a number of open tickets. One particularly interesting ticket related to uncoverable code which gets executed. Devel::Cover marks such code as in error, which reduces the coverage figures. But it turns out there are legitimate use cases where this is unwanted. So I added an option ignore_covered_err which can be applied globally or per uncoverable directive. This option will ensure that covered code never generates an error in the coverage.

Beyond the technical value of the summit, the majority of the attendees are regulars and it was lovely to see old friends again, to chat and catch up, as well as to meet the new attendees. I'm grateful for the COVID policy the organisers put in place which allowed me to safely attend, and for the willingness of the attendees to abide by it. And a special thank-you goes out to the organisers who did a magnificent job despite not having the time to do so, and to all the sponsors, without whom the whole summit would not have been possible.

Booking.com, Deriv, Grant Street Group, Fastmail, cPanel, Perl Careers, MaxMind, Fastly Inc., Perl Maven, OpenCage, Perl Services, Oetiker+Partner, Procura.

This year I was again privileged to attend the Perl Toolchain Summit, held at Bisham Abbey, near Marlow in southern England.

The Perl Toolchain summit is an opportunity for the developers and maintainers of Perl's infrastructure to get together and do whatever is necessary to keep that infrastructure running well and improve its services.

I arrived at PTS with a TODO list which I knew was already more than I would be able to complete. But PTS is not just about getting your own work done. A major benefit of having thirty perl hackers together in a room is that often when someone hits a problem or needs guidance or information they can simply wander over to the table where the author of the software they are using is sitting and ask them directly.

On the first day in the standup meeting I mentioned that I was looking for a solution to install system dependencies of CPAN modules before generating their coverage and that I was sure someone must have solved that problem. Immediately thereafter Slaven came up to me and told me about his module CPAN::Plugin::Sysdeps which does just that. Philippe offered to integrate it into cpancover and before the end of the summit I had a pull request. This type of interaction is a major benefit of the PTS.

Similarly I have (along with many others), for years, wanted to be able to gather coverage information for top-level statements in modules. This is not easy because perl executes these statements on loading the module and then, with no further use for the optree, throws it away, so that Devel::Cover can't get to it later.

This year, I managed to enlist the help of Aaron Crane, who knows far more about the perl internals than I do. Aaron had previously suggested that the DB::postponed subroutine might be part of a solution and this year he spent some time determining the rest of the solution. I was then able to program it up as a proof of concept. The full solution will require changes to the perl core which I will still need to properly implement, but without Aaron's help I'd never have got to a solution. (I know this because I have spent a long time trying and I was on the wrong path(s).)

For a while I have been discussing closer integration of cpancover with metacpan. The fruit of part of these conversations came online a short time ago when metacpan started showing module coverage figures and linking into cpancover for more detailed information. We have also been discussing matters of infrastructure and investigating backup/failover/redundancy for cpancover. Based on some plans made last year and implemented recently, along with Leo's work during PTS, we now have cpancover data backed up to the metacpan infrastructure, and we have a plan for dealing with downtime on the main cpancover server. This is the sort of thing which is so much easier to do by pulling three or four people into a discussion rather than trying to coordinate over mail or IRC.

PTS has become the most important Perl event of the year for me, and I am very grateful to everyone who attends, to the organisers, Neil, Philippe and Laurent, who did a wonderful job, and to all the sponsors without whom it would not have been possible: Booking.com, cPanel, MaxMind, FastMail, ZipRecruiter, Cogendo, Elastic, OpenCage Data, Perl Services, Zoopla, Archer Education, OpusVL, Oetiker+Partner, SureVoIP, YEF.

This year marks the 10 year anniversary of the Perl Toolchain Summit, formerly known as the Perl QA Hackathon.

The Perl Toolchain Summit provides an opportunity for around 30 Perl developers to get together for four days or so to talk about and develop the infrastructure which surrounds Perl. This includes being able to build modules, being able to test them on multiple systems, being able to find out about them, being able to manage them, and many other related areas.

Salve Nilsen started the QA Hackathon back in 2008 in Oslo, and we returned in 2018 with Salve again being involved in the organisation.

This year I was very happy to discover that a number of people arrived in Oslo with the idea of working on some area of Devel::Cover (the Perl code coverage module) or cpancover (the service that provides coverage information for all of CPAN).

This meant that for me much of the summit was spent in collaboration with people who were doing the real work, and I was very pleased about that - such collaboration, which is facilitated and sometimes only made possible by being together physically, is the purpose of the summit.

One such collaboration was with Joel Berger who implemented a queueing system for cpancover.com. Currently I have a crude shell script which does the work it thinks it needs to do, sleeps for 10 minutes and then looks for new work. This means that there can be a delay of a couple of hours or more between a module being uploaded and the coverage being calculated, especially if the module being uploaded happens to have been uploaded just after another module for which the coverage run takes a particularly long time. With the queueing system Joel has implemented the coverage will usually start to be calculated almost immediately. There are still a few nits to be sorted out, but I hope to put this live in the near future.

For quite a while now I have been trying to find a way to get the coverage information from top-level statements in modules. This has become more important as Dancer, Moose and other frameworks have had more code in top-level statements. Aaron Crane spent some time on working out how this might be done and has come up with a solution which is tantalisingly close, but so far we've not quite got to the finishing line. The problem here is that perl, quite reasonably, recognises that this code will only ever be run once, at the start of the execution and so it throws away the optree as soon as it has been executed. The optree is Devel::Cover's hook into what code has been executed, and persuading perl to leave to optree lying around where we can find it later has proven to be remarkably difficult. I will continue looking at this but thanks to Aaron's knowledge and assistance we may shortly* have a solution. *In relation to the length of time I have wanted a solution.

The UI on cpancover.com is awful, in many ways. Babs has be doing wonderful work on making Perl websites look as if they were created this century. We looked at cpancover and she sees a lot of potential for improvement. However, I do not want the front page of cpancover to be a page that is visited very often. Rather, I would prefer that metacpan linked to the internal cpancover pages and thus metacpan becomes the site to search for coverage data.

So I discussed with the metacpan team how we might incorporate coverage data into metacpan, and use its UI as the main starting point for getting access to coverage data. Mickey spent some time building a system to take the data from cpancover and add it into the the metacpan data. The data still needs to be incorporated into the metacpan UI, but we are almost there.

I talked with Garu about uploading users' local coverage data to cpantesters via Test::Reporter and how we might be able to eventually provide coverage data from many users on different systems in consistent, merged coverage reports. There is a lot of work to be done here, but we might be able to get a simple first step in place without too much difficulty . (By we, I mean Garu, of course. And thus I mean without too much difficulty for me.)

As far as my individual work was concerned, I have managed to get the cpancover reports flowing again. I had been storing and serving all the coverage data I had ever calculated, and I managed to run out of inodes on the partition where that data was stored. I have now implemented a retention policy of only serving the latest three module versions, but older versions are being archived. I also managed to bring the server down for a number of hours, so sorry about that if you noticed.

All in all, this was an extremely productive summit for me with respect to Devel::Cover and cpancover. I consider the Perl Toolchain Summit to be the most important event in my technical calendar and I am extremely grateful to the organisers of this year's summit, as I am to the organisers every year, for all their efforts in making it possible. Its also great to see old friends as well as people who have previously only been known online. Along with the work of the organisers, the summit would not have been possible without the generosity of our sponsors. And, of course, the volunteer efforts put in by the talented and diligent attendees whose ranks I somehow managed to infiltrate.

I'm already looking forward to next year.

Sponsors for the Perl Toolchain Summit 2018

NUUG Foundation, Teknologihuset, Booking.com, cPanel, FastMail, Elastic, ZipRecruiter, MaxMind, MongoDB, SureVoIP, Campus Explorer, Bytemark, Infinity Interactive, OpusVL, Eligo, Perl Services, Oetiker+Partner.