Cacheing Plack Proxy - speed up development

At work we scrape data from a slow government website (it's public data and you are allowed to use it, but they don't have a way to download it!).

We thought if only there was a quick way to cache the results as we develop the code...

A CPAN search later... we want a proxy and something to cache

use Plack::Builder;
use Plack::App::Proxy;
use Plack::Middleware::Cache;

my $app
    = Plack::App::Proxy->new( remote => "" )->to_app;

builder {
    enable "Cache",
        match_url => '^/.*',               # everything
        cache_dir => '/tmp/plack-cache';

Save this as app.psgi, run plackup and point our code to http://localhost:5000/ and it's all working!


Very clever. :)

WWW::Mechanize::Cached might work for you too, but this is very shiny. :)

Wrap that in LWP::Protocol::PSGI and you can run the code against the actual service URL instead of localhost.

Thanks a lot for the LWP::Protocol::PSGI tip.

I just came up with a transparent Twitter API caching solution by combining Plack::App::Proxy, Plack::Middleware::Cache and LWP::Protocol::PSGI.


Leave a comment

About Ranguard

user-pic London Perl developer