Reducing Database Backups
Having monitored the logs on the server over the last few days, I've noticed some improvements that can be made to some of the processing. Some of the improvements are in the server code itself and will be introduced alongside some of the changes towards CPAN Testers 2.0.
However, some of the proposed changes are to the files available on the Development site. Specifically the database downloads. As there hasn't been anyone accessing any of the CSV files for the Uploads DB, these will be removed next week. Of the other files, although the GZip and BZip files are accessed, they aren't accessed very often, except for the cpanstats DB. As such the archives for both the Uploads DB and Release DB will only be updated once a day from next week.
These changes are to help reduce the processing load on the server, which in turn will help to improve response times for web pages and allow more processing capacity for parsing reports once we move to the Metabase.
This is the new account for incidental and summary updates to what's happening with the CPAN Testers. For all the latest news and views please see our blog.
The uploads database is probably my Top 100 stuff feeding on it.
Can you let me know when the cron timings are for these updates, so I can make sure my update only runs once a week on the day immediately after it (so the stats dump doesn't start providing information for modules that don't exist in the uploads dump).