How can I grep all of BackPAN?
How can we make a Perl code search so I can grep all of CPAN? I would have done this with the now-dead Google Code Search, which used to make this part of world's information and universally accessible and useful.
Specifically, I want to look at every instance of META_MERGE in Makefile.PL in every distribution in BackPAN. I can easily program this task with my DPAN stuff since I already have a way to crawl CPAN and look in every distribution. That would take a couple of days to go through 250,000 files (although maybe I should try this with Archive::Extract::Libarchive, which speeds up the main bottleneck in this technique.
I thought about GitPAN for a few seconds, but I want to search things that are not in HEAD, too.
MetaCPAN API might be able to do it, but I don't think I can search on information that isn't already indexed. I can search most of the meta-things about Perl distributions, but I want to use a regex across all files in BackPAN.
Sometimes I've thought about a PPI-based search engine where you could search by what something is. For instance, I want to find all subroutines named
log instead of just searching for the text "sub\s+log" and so on.
I'm sure someone in Perl-land already does something similar with all the right technologies that we could use to make this available to everyone.
I could just unpack all of BackPAN and use `find` and `grep`. That might actually be the easiest way to do it locally, where "easiest" is the least work for me to answer this single question. I'd have to wait a bit, but all that stuff is asynchronous.