Mass-Testing Dancer's Plugins

cross-posted from dams blog

Mass-Testing Dancer's Plugins

So, as I said at YAPC::EU 2012, one thing that remains to be done before Dancer 2 can be released : migrating the plugins, making sure they work with it.

To be able to do that, what's best than an automatic testing facility ?

The goal is to get all Dancer plugins, test them with Dancer1, and Dancer2, and produce a report, to check which one fails and need fixing.

Step 1. Get the list of Dancer plugins.

Easy ! let's use Metacpan. After searching, I finally got a way to get the list of all modules that depend on Dancer. Then filtering out the ones that don't contain "Plugin" will do the trick.

#!/usr/bin/env perl

use Modern::Perl;
use ElasticSearch;

my $es = ElasticSearch->new( servers => '', no_refresh => 1 );

my $scroller = $es->scrolled_search(
    query       => { match_all => { } },
    search_type => 'scan',
    scroll      => '5m',
    index       => 'v0',
    type        => 'release',
    size        => 100,
    filter => {
            term => {
                     'release.dependency.module' => 'Dancer'


my $result = $scroller->next;

my %plugins;
while ( my $result = $scroller->next ) {
    $result->{_source}->{name} =~ /Dancer-Plugin/
      or next;
    my $name = $result->{_source}->{name};
    $name =~ s/-\d.*//;
    $name =~ s/-/::/g;
    $plugins{$name} = 1;
say $_ foreach sort keys %plugins;

Cool, let's save this script as

Step 2. Prepare two Perl environments

We want two instance of Perl, without polluting anything. We'll use perlbrew for that. Easy. The small trick is to have PERLBREW_ROOT initialized to a local directory to not polute existing ~/perl5 installation.

Step 3. Have a way to test modules

Well, let's use cpanm, which has an option --test-only to only test a module without installing it. Oh but plugin modules that we'll test may require dependances. We'll install them using cpanm --installdeps, which does just that.

Step 4. Create a result file

I was lazy and just output to a .csv text file, but I may store the results somewhere else later.

Step 5. Let's glue all that in a Makefile

The beauty of this is that by writing a proper Makefile, we can install this auto-tester anywhere. The requirements are minimal : bash, curl, and perl, with ElasticSearch and Modern::Perl installed.

( This post is intentionally named the same as the one from Yannick, because he proposed a different implementation. We'll try to merge the good ideas together )


# set up the local Perlbrew location
    && source ${PERLBREW_DIR}/etc/bashrc \

# This will run the cpanm from the local Perlbrew, on each of the 2 perls
CPANM_DANCER1=${PERLBREW} exec --with dancer1_plugin_tests ${PERLBREW_DIR}/bin/cpanm
CPANM_DANCER2=${PERLBREW} exec --with dancer2_plugin_tests ${PERLBREW_DIR}/bin/cpanm

# loop on the plugin list, install the deps and test each plugin and output result
all: got_curl got_bash got_perlbrew_perl plugins_list
    echo "Plugin name,Pass on Dancer 1,Pass on Dancer 2" > result.csv
    @for i in `cat plugins_list`; do pass_d1=0; pass_d2=0; \
      echo " ---------- TESTING on Dancer 1 : $$i"; \
          ${CPANM_DANCER1} -n --installdeps $$i \
              && ${CPANM_DANCER1} --test-only $$i && pass_d1=1; \
      echo " ---------- TESTING on Dancer 2 : $$i"; \
          ${CPANM_DANCER2} -n --installdeps $$i \
              && DANCER_FORCE_PLUGIN_REGISTRATION=1 ${CPANM_DANCER2} --test-only $$i \
              && pass_d2=1; \
          echo "$$i,$$pass_d1,$$pass_d2" >> result.csv; \

    rm -rf plugins_list

    rm -rf ${PERLBREW_DIR}

# Get and install Perlbrew locally
    @echo " - creating a local perlbrew"
    export PERLBREW_ROOT=${PERLBREW_DIR} && curl -kL | bash

# Get and install cpanm in the local Perlbrew
    ${PERLBREW} install-cpanm

    @which curl >/dev/null \
        || ( echo "you don't have curl, please install it and retry" && false )

    @which bash >/dev/null \
        || ( echo "you don't have bash, please install it and retry" && false )

got_perlbrew_perl: ${PERLBREW_DIR}/bin/perlbrew ${PERLBREW_DIR}/bin/cpanm perlbrew_dancer1_plugin_tests perlbrew_dancer2_plugin_tests

# Build perl from scratch and call this instance dancer1
    ${PERLBREW} list | grep dancer1_plugin_tests > /dev/null \
        || ( ${PERLBREW} install -j 2 -n perl-5.16.1 --as dancer1_plugin_tests )

# Build perl from scratch and call this instance dancer2
    ${PERLBREW} list | grep dancer2_plugin_tests > /dev/null \
        || ( ${PERLBREW} install -j 2 -n perl-5.16.1 --as dancer2_plugin_tests \
             && ${CPANM_DANCER2} ${HERE}/../.. )

# This gets the list of plugins, as previously described
    ${HERE}/ > plugins_list

It's really cool to see what you can do with modern tools of the Perl ecosystem !

About Damien "dams" Krotkine

user-pic I blog about Perl.