CPAN Testers

With the big announcement on Monday, that we are now live with CPAN Testers 2.0, the next couple of months are likely to be interesting to say the least. We now have the task of helping people set up their smoker clients to use the HTTP submission process, rather than the current SMTP defaults. In most cases it will just require some changes to existing configuration settings and the possible upgrade to the latest smoker tools. Part of the setup process does involve registering with the Metabase submission service, which initially will be a little bit of a manual process, although will be a more automated process once the web site is up and running. Once correctly configured, you should then be able to continue testing as before.

The beginnings of some howto guides are now available on the CPAN Testers Wiki, which we'll elaborate on over the coming weeks. In the first instance the plan is to move the most prolific testers across to the new submission process. In several cases, these guys were beta testers for us, so are already setup. Those that haven't registered will initially need to contact David Golden, although if you submitted less than 100 reports in June, please wait until the web service is online. The registration process provides a "profile file" that contains your identifying credentials. Using the new web service, existing testers will be able to register and claim addresses which they have previously used to submit reports, and associate them with your "profile", so that you can continue to be identified in the CPAN Testers Leaderboard. The web service will be deployed very shortly, so please be patient, while we get the top testers setup first.

Once the bulk of testers are using the HTTP submission process, the next step will be to contact the less frequent testers. This will mostly be a case of tailing the SMTP submissions and contacting testers individually where possible. The SMTP service will be shut down on or around 31st August, so there should be plenty of time for testers to migrate their current smoker clients to the new system. After this date any report submitted via SMTP will not be processed.

Over the next few months the smoker client tools will be upgraded to use the HTTP submission process by default rather than the SMTP process, thus allowing new testers to automatically be setup to use the new system. In addition, the intention is to provide a mechanism to use the registration web service via an automated API, which will mean that Strawberry Perl installations can automatically be setup to run a CPAN Testers client, and be pre-registered without the user having to do this themselves. The downside to this is that the profile will not be tied to a user that we can necessarily contact, however there are already several submissions each month that use fictious email addresses. The difference with the new registration process, is that at a later date the tester can more actively use their profile file to claim a real email address. Over time we hope this will then become a much more reliable service.

As mentioned in the press release, CT2.0 offers a few advantages for us, one of which is that it doesn't truncate reports. Although, we will be monitoring the reports submitted to ensure that none are unnecessarily large. It does mean that we should be able to parse all the necessary metadata from the report, reducing the number of submissions with conflicting or missing data. As we begin to provide more structured data within the report submission, validation of reports will become much more reliable, and include many more detailed aspects of the testing environment.

One aspect of the testing environments, that often causes confusion, is the operating system used. Currently our understanding of the OS used is mostly drawn from the 'Perl -V' configuration settings, which may not always be correct. In addition, for some OSes the information provided doesn't give a clear enough picture. For example, if the OS is Linux, knowing whether it's RedHat, Debian or SUSE based may have significant benefits to fixing problems, likewise differentiating between Tiger, Leopard and Snow Leopard for Mac OSX. As such, a Birmingham.pm initiated project aims to provide this information as part of the structured data of a report. As a first step the creation of Devel-Platform-Info is underway, and will provide the metadata required in a consistent format, regardless of platform. This will eventually be then included in the CT2.0 style of reporting, simply as an additional Fact.

We've again had some run-ins with a certain search engine last month, and despite various erroreous claims, we were able to provide logs that disproved them all. The search engine has now finally disabled the aggressive crawling of the CPAN Testers site ... having only taken 6 months of asking! They have asked to be allowed to crawl the website again, having claimed to have fixed the problems, but until I have some free time to properly monitor their bots, they will have to wait.

As mentioned in the press release on Monday, there are two forthcoming conferences which will feature talks concerning CPAN Testers. David Golden will be presenting "Free QA! What FOSS can Learn from CPAN Testers" at OSCON and Barbie will be presenting "CPAN Testers 2.0 : I love it when a plan comes together" at YAPC::Europe.

After several long months of getting CPAN Testers ready for launch, the next month is likely to see activity centred around minor fixes, howtos and helping people migrating to CPAN Testers 2.0. If you do hit any problems, please post them to the cpan-testers-discuss mailing list, where many of the high profile testers and developers reside. After the deadline, we'll be concentrating on some of the new systems that have been waiting in the wings, and taking a more detailed look at providing more structure data in the reports. A new dawn rises.

Cross-posted from the CPAN Testers Blog.