Speeding up Test::WWW::Mechanize tests with clone()

Today I was looking at old automated Perl test based on Test::WWW::Mechanize. It was it testing a complex form. For each of about 10 tests, it loaded the form (with a get_ok() call), and then submitted the form with a variety of input.

Now that we run about 25,000 tests in total in the test suite, I’m always looking for ways to speed up the tests. HTTP calls are relatively slow, so systematic ways to slim them down are attractive.

In this case, I found there was a simple change that I could apply that sped up the particular test by about 28%.

Each time the test was calling get_ok() on the page, it was getting back the same result, which is wasteful. I refactored it like this:

my $base_mech = Test::WWW::Mechanize->new;

Then, everywhere we had a get_ok() call to load the page again, I replaced it with this:

$mech = $base_mech->clone;

Both approaches gave me a Test::WWW::Mechanize object in just the state I desired, but the new approach was must faster.

An alternate approach in a case like that is to submit the form just once to make sure that you’ve got all the form field names correctly matching up with the processing URI that it submits to. After that, instead of “GETing” the form again and calling submitformok(), just call post_ok() to POST directly to the processing run mode with the appropriate values.

Either approach avoids a lot of unnecessary GET requests in your test suite.

For more tips on speeding up your test suites, come to my talk on topic of speeding up test suites in YAPC in June, 2013 in Austin, Texas.

Thanks to Olaf Alders for HTTP::CookieMonster

The old, horrible way, to get a cookie by name with LWP::UserAgent or WWW::Mechanize:

my $cookie = $a->cookie_jar->{COOKIES}->{".$SITE_DOMAIN"}->{'/'}->{$COOKIE_NAME};

The new way, with HTTP::CookieMonster:

my $monster = HTTP::CookieMonster->new( cookie_jar => $a->cookie_jar );
my $cookie = $monster->get_cookie($COOKIE_NAME);

Much better. Thank you. I’d love to see something like this built right into WWW::Mechanize.

About Mark Stosberg

user-pic I help maintain CGI::Application, CGI.pm, CGI::Session and Data::FormValidator. I'm interested in balancing technology with simplicity and sustainability.