CPAN Testers

If I could kick the Amazon SimpleDB we have, I'd take great delight in doing so. Thankfully, our days of relying on it are numbered ... in a good way. More news on this will be forthcoming soon.

However, over the last month the most popular email regarding CPAN Testers I have received, was along te lines of "is CPAN Testers broken?" The answer is definitely no, but Amazon's SimpleDB most definitely is. It's not clear why its broken, but the results it returns often mean we don't get the full result set requested, and the cpanstats database appears to miss reports. September appears to have been particularly bad with 496784 reports missing from the initial result set. As such, I have been running a catch-up process for the past few weeks, and it is picking up all the previously missed reports. I'm now running this for October, and over the next week, possibly two, I aim to be completely up to date. Thanks also to some help and ideas from David and Andreas, I now have a process ready to deploy that will allow the system to run a catch-up daily, and avoid missing reports entirely.

According to SimpleDB the following is an ordered result set by date:

2012-10-10T11:02:17Z
2012-10-10T11:02:20Z
2012-10-10T08:09:10Z
2012-10-10T08:07:59Z
2012-10-10T11:02:12Z
2012-10-10T11:02:30Z
2012-10-10T11:02:23Z
2012-10-10T11:02:34Z
2012-10-10T11:02:38Z
2012-10-10T14:14:54Z
2012-10-10T11:02:46Z
2012-10-10T11:02:41Z
2012-10-10T14:14:55Z
2012-10-10T08:09:33Z
2012-10-10T11:02:47Z

Is it any wonder that we miss so many reports!

Part of the solution has been to look at a slightly different query. The tail log that David currently creates unfortunately only lists a reference to the underlying report summary fact, and not the main report fact. As such, I have now adjusted the catch-up to use the same query as the tail log. Once in place I should be able to have a much more reliable feed.

On the mailing list, Shlomi Fish asked about Getting rid of the problem of an overloaded PERL5LIB on Windows, Kjetil Kjernsmo asked about Scalar::Util 1.23 openhandle changes and several people asked about missing reports.

Rest assured I am working on the missing reports and hope to have everything back on track within the next week or two.

Cross-posted from the CPAN Testers Blog.