Check your test coverage with Devel::Cover

For CPAN Day, you could check the test coverage for your distributions using Devel::Cover. If your distributions are already covered, or you don't have any distributions, take a look at the coverage results for some other dists on CPAN Cover, and maybe you could submit something for someone else's dist.

Devel::Cover checks how thoroughly your dist's testsuite exercise your code, and also reports on how well your documentation covers your public interface.

Running it is pretty easy. If your dist uses ExtUtils::MakeMaker, then in the top directory of your dist, run:

% perl Makefile.PL
% make
% cover -test

For Module::Build users:

% ./Build testcover

And for Dist::Zilla users:

% dzil cover

This runs your testsuite, collecting coverage stats as it does. At the end it will display a summary. Here's the results for Lingua-EN-Numbers:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
.../lib/Lingua/EN/Numbers.pm   99.0   81.9   61.9  100.0    0.0  100.0   85.6
Total                          99.0   81.9   61.9  100.0    0.0  100.0   85.6
---------------------------- ------ ------ ------ ------ ------ ------ ------

You'll notice that it also generates a cover_db directory, where it writes everything. The thing you want to look at is coverage.html, which has the info displayed to STDOUT, and a little bit more:

Read the documentation, including Devel::Cover::Tutorial, but here's a quick summary of the four types of code coverage:

  • stmt or Statement: what percentage of individual perl statements are exercised?
  • bran or Branch: what percentage of the clauses in if, else and elsif statements are covered.
  • cond or Condition: are you exercising all permutations of your conditional expressions? Eg given if ($x || $y) do you have tests where $x is true and $x is false?
  • sub or Subroutine: are all subs in your code exercised?

Clicking on each individual coverage figure takes you to a page which shows the lines of code from the relevant module, and coverage details. Here's the start of the branch coverage details for Lingua::EN::Numbers

That's telling me I don't have any tests where DEBUG is true.

At the simplest level, using Devel::Cover improves the coverage and thus quality of your test suites. But I've found that when you start using it, it changes how you think about and write your test suites, and then starts subtly changing how you think about and write your modules too.

Paul Johnson, who wrote and maintains it, is giving a tutorial on Devel::Cover at the Swiss Perl workshop, on the 6th September. If you can't attend that, have a read of Paul's slides from the Swiss Perl Workshop 2013.

Update

cono wrote about using Devel::Cover with Dist::Zilla, for vim users.

3 Comments

Just great - many thanks!

Just started using this last week and its great. Even integrated it in to our automated build script, so it runs every commit.

Leave a comment

About Neil Bowers

user-pic Perl hacker since 1992.