web scraper Archives

Scrappy - Automated Full Service Web Spider

For the past two/three months (on and off) I have been developing a web spider framework that would bring together many web scraping concepts and methodologies and combine them in one library (package). That framework is Scrappy which is a play on the term (scraper happy or happy scraper), not to be misunderstood as a port of Scrapy the Python Scraper Framework.

For the sake of time I will just highlight some of the features that I think are innovative and add value.

Scrappy can be used as DSL or OOP (even interchangeably). e.g.
[oop]
use …

About awnstudio

user-pic Al Newkirk is a web application developer from Philadelphia, PA in the United States who specializes in Perl development with MySQL on a Windows platform using Strawberry Perl. I do a bunch of other stuff too, stick around and find out what!