February 2013 Archives

Web Scraping with Modern Perl (Part 2 - Speed Edition)

tl;dr

Grab the gist with the complete, working source code. Benchmark it against the one featured on the previous article:

$ \time perl mojo-crawler.pl
23.08user 0.07system 2:11.99elapsed 17%CPU (0avgtext+0avgdata 84064maxresident)k
16inputs+0outputs (0major+6356minor)pagefaults 0swaps

$ \time perl yada-crawler.pl
8.83user 0.30system 0:12.60elapsed 72%CPU (0avgtext+0avgdata 131008maxresident)k
0inputs+0outputs (0major+8607minor)pagefaults 0swaps

How can it be 10x faster while consuming less than a half of CPU resources?!

About stas

user-pic Just another lazy, impatient and arrogant IT guy.