CPAN Testers Server Update - 20/09/2011
The MySQL databases have now been rebuilt and correctly synced between each other. The SQLite databases are now being updated and these should be completed by the end of the week. This has enabled me to write some simple scripts to create and repair the databases, which I'll now be including in a separate distribution to be released on GitHub. It will also include all the apache, mysql, logrotate, cron and other config and script files, so that if we ever have to rebuild again, getting started is a lot easier.
The websites are now rebuilding too. The statistics site will be complete shortly, as it takes a while to rerun all the statistical analysis. Trying to analyse everything all at once tends to grind the server to halt, whereas analysing bitesize chunks, although slightly slower, uses less memory and saves progress to disk, so we don't have to start again from scratch if anything falls over.
The reports site still has 27,000 entries to process, so I'm not expecting that to be down to a more reasonable number until the weekend. Turning the builder back on has highlighted some of the tweaks that are now missing from the work I did in the few weeks prior to the disk crash. In time I hope to address these performance tweaks again, but for now I'm just letting it build in peace :)
I'm now preparing to turn back on the feeds, to catch up on all the reports submitted in the last few weeks. I have run a couple of requests to fill in some of the gaps, and all seems to be fine, so I hope that a few days of dedicate feed processing should get through the bulk of reports from the last 3 weeks.
I haven't switched apache back on yet, as one thing that I've noticed is that server load is getting unnecessarily high. I know there is an intense amount of disk IO happening at the moment, but there appears to far too much disk access that there was previously. As a consequence I've started to reduce both the number of logs and log output. In most cases the output was only for information or debugging, so isn't necessary except in unusual circumstances. Hopefully after the weekend I'll get to switch Apache back on and can start some performance tuning.
If anyone has knowledge of performance tuning Debian Squeeze system processes, please get in touch. Trying to work out what tasks are essential and what can be run less frequent is getting tricky.
Cross-posted from the CPAN Testers Blog
This is the new account for incidental and summary updates to what's happening with the CPAN Testers. For all the latest news and views please see our blog.
Nothing here just saying thanks for your work!
I hope you success with Debian, I cant really help much more than hope you best luck. I will let you know if get touch to someone with better Debian knowledge
I can help. I'm a linux admin who mostly uses debian. I'll be lurking in #cpantesters if you want.
BTW will these scripts one day form some kind of a package to make it easy for someone to setup a CPAN Testers database for in-house modules at a company?
That might be interesting too to some people.
@Gabor, they will be a starting point at the very least, as some of the configuration files will be very specific to the setup we have. However, I do plan to document the processes involved, so that anyone wanting to replicate this for their own testing will know what is and isn't appropriate for them.