Most Popular Testing Modules - January 2010

Back in September 2008, I had a list of the most popular testing modules on the CPAN. I created this list because I was writing Test::Most and I needed to know them. Tonight, after hearing that the next Perl-QA hackathon is in Vienna, I thought about what I might want to accomplish and decided to see if the most popular modules had changes. In 2008, out of 373 test modules, we had the following top 20:

 1  Test::More                          69396
 2  Test                                10912
 3  Test::Exception                     2314
 4  Test::Simple                        962
 5  Test::Base                          610
 6  Test::NoWarnings                    386
 7  Test::Builder::Tester               308
 8  Test::Deep                          254
 9  Test::Pod                           223
10  Test::Warn                          217
11  Test::Differences                   213
12  Test::MockObject                    187
13  Test::Pod::Coverage                 157
14  Test::Builder                       131
15  Test::WWW::Mechanize::Catalyst      118
16  Test::XML                           113
17  Test::Block                         112
18  Test::Perl::Critic                  110
19  Test::Distribution                  107
20  Test::SQL::Translator               101

In January 2010, out of 493(!) Test:: modules, we have the following top 20:

 1  Test::More                          86194
 2  Test                                10789
 3  Test::Exception                     3548
 4  Test::Base                          1006
 5  Test::Simple                        962
 6  Test::NoWarnings                    884
 7  Test::Deep                          413
 8  Test::Warn                          350
 9  Test::Pod                           335
10  Test::Builder::Tester               322
11  Test::Differences                   311
12  Test::Pod::Coverage                 277
13  Test::Perl::Critic                  256
14  Test::MockObject                    205
15  Test::Builder                       203
16  Test::Most                          170
17  Test::Block                         169
18  Test::WWW::Mechanize::Catalyst      162
19  Test::Distribution                  150
20  Test::Kwalitee                      143

That's when I noticed something very interesting. My Test::Most module, the one I created this list for, was already in the number 16 slot (it's currently at the number 13 slot if you ignore modules in the Test:: namespace but which don't export test functions)! That's when my failure dawned on me. Given that people who use Test::Most are more serious about testing, it's reasonable that they're more serious about testing, thus artificially inflating my stats. I was counting the number of times the modules were being used. However, if I write, say, an extra 160 test programs for my modules, each of which uses Test::Most (or indeed, just type use Test::Most 160 times in a single module), then I could push my module into the top ten. So I reran my stats per distribution:

1   Test::More                          14111
2   Test                                 1736
3   Test::Exception                       744
4   Test::Simple                          331
5   Test::Pod                             328
6   Test::Pod::Coverage                   274
7   Test::Perl::Critic                    248
8   Test::Base                            228
9   Test::NoWarnings                      155
10  Test::Distribution                    142
11  Test::Kwalitee                        138
12  Test::Deep                            128
13  Test::Warn                            127
14  Test::Differences                     102
15  Test::Spelling                        101
16  Test::MockObject                       87
17  Test::Builder::Tester                  84
18  Test::WWW::Mechanize::Catalyst         79
19  Test::UseAllModules                    63
20  Test::YAML::Meta                       61
21  Test::Synopsis                         57
22  Test::Compile                          56
23  Test::Portability::Files               54
24  Test::Most                             49

OK, so I dropped down to a more reasonable 24th place. But a quick check of CPANTS showed that I wasn't finding as many uses of Test::Most as I should because there were cases where:

There was also the case where it turns out that Robert Krimen is a huge fan of Test::Most (Hi Robert!) and has enough modules out there to artificially inflate my stats. So here's yet another list, showing test modules per author:

1   Test::More                          2669
2   Test                                 685
3   Test::Exception                      260
4   Test::Simple                         161
5   Test::Pod                             76
6   Test::Warn                            66
7   Test::Perl::Critic                    65
8   Test::Deep                            60
9   Test::Pod::Coverage                   55
10  Test::NoWarnings                      51
11  Test::Differences                     50
12  Test::Builder::Tester                 49
13  Test::MockObject                      49
14  Test::Base                            47
15  Test::WWW::Mechanize::Catalyst        36
16  Test::Builder                         27
17  Test::Kwalitee                        26
18  Test::MockObject::Extends             24
19  Test::Distribution                    21
20  Test::Output                          19
21  Test::XML                             17
22  Test::Moose                           16
23  Test::Harness                         16
24  Test::LongString                      15
25  Test::Tester                          13
26  Test::Spelling                        12
27  Test::MockModule                      11
28  Test::UseAllModules                   10
29  Test::TCP                             10
30  Test::Most                            10

That seems a touch more reasonable, though I confess I'd like to see more folks using this module. It really does make your testing life simpler.

And as for the beautiful Test::Class which more people should be using:

  • 95 test programs (25th place)
  • 19 distributions (42nd place)
  • 9 authors (33rd place)

Come on, folks! You can do better than that, right?

1 Comment

There's also the fact that a lot of authors don't specify modules for "author tests" (like Test::Perl::Critic, Test::Portability::Files, etc...) in their .PL files, but just use them if available. I'm one of those. Would the way you collected the data have caught that?

(The idea behind that is, why make people install them if they don't need them unless AUTOMATED_TESTING or RELEASE_TESTING is set?)

Leave a comment

About Ovid

user-pic Have Perl; Will Travel. Freelance Perl/Testing/Agile consultant. Photo by http://www.circle23.com/. Warning: that site is not safe for work. The photographer is a good friend of mine, though, and it's appropriate to credit his work.