Cacheing Plack Proxy - speed up development
At work we scrape data from a slow government website (it's public data and you are allowed to use it, but they don't have a way to download it!).
We thought if only there was a quick way to cache the results as we develop the code...
A CPAN search later... we want a proxy and something to cache
use Plack::Builder;
use Plack::App::Proxy;
use Plack::Middleware::Cache;
my $app
= Plack::App::Proxy->new( remote => "http://london.pm.org/" )->to_app;
builder {
enable "Cache",
match_url => '^/.*', # everything
cache_dir => '/tmp/plack-cache';
$app;
};
Save this as app.psgi, run plackup
and point our code to http://localhost:5000/ and it's all working!
Very clever. :)
WWW::Mechanize::Cached might work for you too, but this is very shiny. :)
Wrap that in LWP::Protocol::PSGI and you can run the code against the actual service URL instead of
localhost
.Thanks a lot for the LWP::Protocol::PSGI tip.
I just came up with a transparent Twitter API caching solution by combining Plack::App::Proxy, Plack::Middleware::Cache and LWP::Protocol::PSGI.
Awesome.
@aristotle - yea, the LWP::Protocol::PSGI tip is really nice :)