CPAN modules for making HTTP requests
I've published a review of CPAN modules for making HTTP requests. This covers 20 different modules from 19 distributions. I focussed on making GET and POST requests, whether the module supports https and redirects, and some other features.
As ever, I've no doubt missed a couple — let me know if you're aware of one not on the list. Also happy to hear if there are other aspects you think should be included in the comparison.
Perl hacker since 1992.
Regarding the "why" of Mojo::UserAgent in comparison to LWP:
http://mojocasts.com/e5
The feature-set speaks for itself.
@tempire: is there a version for people who don't watch podcasts?
I think the biggest argument for Mojo::UserAgent is that it integrates with Test::Mojo to easily allow for testing of a Mojolicious app. I'm nearly sure that that was why it was created.
It does do lots more though, including full HTML/XML DOM parsing with CSS3 selectors and even JSON all built in. Of course these were included to help testing, but they become useful outside the Mojolicious world. Even more, the whole system is non-blocking (if used with callbacks).
I can say, at YAPC::NA Glen Hinkle was asked why, since Mojo::UA is so awesome, hasn't it been extracted from the Mojolicious system and released to the CPAN separately and he said that though it had been discussed, there are two reasons that it won't be. First, the underlying system is deeply tied to Mojolicious's event loop and other internals so it would be hard to extract. Second, the whole system is so lean that it that its not an undue burden to install the whole thing just to the UA. It installs on basic hardware in about a minute (that a design goal!) and with `HARNESS_OPTIONS=j9 cpanm Mojolicious` on good hardware it installs in about 15s! I know that at least one prominent Perl author uses M::UA often outside of any Mojolicious app.
That's a great list, thanks for sharing.
For completeness, it'd be good to see some of the event-based modules in there - personally I use Net::Async::HTTP these days, but a quick search shows these as well:
There's also a few special-case options such as LWPx::ParanoidAgent or HTTP::Async.
You wish for an encode_form_data function. It is already exposed as part of a LWP dependency:
Thanks Tom, I'll include those in the next version.
Thanks Glen, I'll look at that when I'm back off holiday, and update the Mojo::UA section as needed for the next version.
I'll include that option in the next version, and will also benchmark against a simple function. Your approach would really increase the dependencies for some modules, I suspect.
Indeed, that's a great summary Joel. There are also collections (not mentioned @ YAPC) and out-of-band debugging:
Collections
Navigate though the dom using functional chains:
my @elements = $ua->get('mojolicio.us')
->res->dom->find('#mojobar-links > a')
->slice(2..7)
->grep(sub { return 1 if shift->{href} =~ /github/ })
->each(sub { say shift })
->reverse;
Video: http://mojocasts.com/e5#Collections
Docs: http://mojolicio.us/perldoc/Mojo/Collection
Out-of-band debugging
Monitor the web request on stderr without touching code at all:
MOJO_USERAGENT_DEBUG=1 perl my_script.pl
https://gist.github.com/3180250