August 2010 Archives

Testing that darned filehandle

It wasn't fun finding out that some legacy code creates a bunch of filehandles on the fly to do logging.

package Some::Client;
use Some::Logging::Module;

# later ...
print MAGICLOGHANDLE $some_data;

That MAGICLOGHANDLE is dynamically generated and shoved into my namespace (along with a ton of other handles, one per log). So I needed to test what was being printed to it. Fortunately, Perl's dynamism makes this really, really easy.

Prime Factors in Perl 6

There's really nothing extraordinary about this and certainly nothing specific to Perl 6, but I really liked how it felt writing code to find prime factors. I do note, however, that the habit of using a dash instead of an underscore in identifiers does slow down the Perl 6 to Perl 5 translation (since I'm doing that manually). Hopefully we won't need to keep doing that translation in the future.

sub prime-factors(Int $number is copy) {
    # don't try to factor prime numbers
    return { $number => 1 } if is-prime($number);

    my %factors;
    for @PRIMES -> $prime-number {
        last if $prime-number ** 2 > $number;
        while $number % $prime-number == 0 {
            %factors{$prime-number} //= 0;
            %factors{$prime-number}++;
            $number /= $prime-number;
        }
    }
    %factors{$number}++ if $number != 1;  # we have a prime left over
    return %factors;
}

for 17, 53, 90, 94, 200, 289, 62710561 -> $number {
    say "Prime factors of $number are: ", prime-factors($number).perl;
}

The is-prime sub merely relies on a list of the first 1000 prime numbers (kept in the @PRIMES array). It was much faster that way. If you desperately must see it, it's in the extended part of the blog entry.

It takes almost 7 seconds to run on my MacBook Pro, but .007 under Perl 5. I think I'll keep this one around for a while to see Rakudo benchmark improvements. As usual, code improvements welcome.

What to know before debating type systems

Originally located at http://www.pphsg.org/cdsmith/types.html, this article explained some basic concepts of type systems. Unfortunately, that page is gone and I had to fetch it from the web archive of that page. The note at the bottom states that contents are in the public domain, so I think it's OK to reproduce here.

What follows is a short, brilliant introduction to the basic concepts of type systems, by Chris Smith.


Filtering Subtests?

I wrote about speeding up Test::Class with subtests. That's worked extremely well for us, but it's raised an interesting problem: how can I filter subtests the way I can filter test methods? Explanation follows, suggestions welcome.

"Pretty" SQL output on test failure

Quite often when running tests, you do something like this:

use Test::Most;

# do stuff

eq_or_diff $have, $want, '... and we gots whats we needs'
    or show $sql, $arguments;

Except that this likely will give you something like this:

# $sql = 'SELECT `format`.`id`, `format`.`title`, `format`.`title_cy`, `format`.`title_gd` FROM `format_container` `me` LEFT OUTER JOIN `format` `format` ON ( `format`.`id` = `me`.`format_id` ) WHERE ( `me`.`container_id` = ? )';
# $arguments = '\'c_series\'';

... and that's a simple one. I didn't like that, so I fixed it.

Speeding up the test suite with subtests

The BBC team I'm currently working on has a very, very slow test suite. On my box, it was generally taking about 2h10m to complete. Between Johan Lindström and myself, we've shaved half an hour from that. Johan used transactional savepoints for his part. This allowed us to roll back some database changes rather than rebuild the database tables. My part involved subtests and a very strange use of Test::Class.

Speeding up the test suite with subtests

The BBC team I'm currently working on has a very, very slow test suite. On my box, it was generally taking about 2h10m to complete. Between Johan Lindström and myself, we've shaved half an hour from that. Johan used transactional savepoints for his part. This allowed us to roll back some database changes rather than rebuild the database tables. My part involved subtests and a very strange use of Test::Class.

Roles in other languages?

In a few days, I'll be giving a talk on roles at DevTank. The audience will be developers (and business people) from many different language backgrounds. I've tried assessing state of the art for roles/traits in other languages, but am falling short. I just don't know other languages/cultures well enough.

I might add that I was disappointed to learn that Scala's traits are just mixins. I wonder how they came up with that name. Surely it wasn't just misunderstanding Smalltalk traits?

Update: As preparation for the talk, I've written a short language-agnostic introduction to traits/roles. I put that on my personal blog instead of this one as I don't want people to see the blog URL and dismiss it as just a "Perl" thing.

Test::Class::Profile

I need to do two things:

  • Think of a better name than Test::Class::Profile
  • Write it

The problem: running several thousand tests in one Test::Class process can swallow up a lot of information. I want to instrument Test::Class to optionally capture that information to assist debugging/management of your test suite. I need to write code which will:

  • Tell you what classes ran and in which order
  • How long did each class take to run
  • How long does each test control method take to run
  • Overhead on test methods from test control methods

The last one is rather interesting. Consider a test class with a setup method which takes 8 seconds to run. If it has 10 test methods, than that's an extra 80 seconds of overhead you've added. If you can get the setup method to run in 4 seconds, you can save 40 seconds on the test class. Do this with just a few test classes and you're talking about serious performance savings.

What other information would you want in something like this? How should the information be recorded/presented?

Your Test Suite is Broken

Here are some indicators that your test suite is broken:

  • It has any failing tests.
  • It emits anything other than test information.
  • It takes too long to run.

People are going to have issues with the second item on that list but they're definitely going to argue with that last one. I don't care. It's time to throw the gauntlet down. Let's look at these three points in detail.

Open Perl modules with vim

After reading about opening Perl module via name with vim, I realized it had a couple of (for me) limitations. First, I always operate from my top-level directory in a project, but that post doesn't prepend a lib/. Second, when I'm running a test suite, I see failure which might be from a module in the lib/ directory, but more often than not are in the t/lib/ directory. Since I keep all of my project directories identical, I can use one little bash function to handle this:

function vim {
  set -- "${@//:://}"
  module="lib/${@/%/.pm}"
  testclass="t/${@/%/.pm}"
  if [ -f $module ]; then
    command vim $module
  elif [ -f $testclass ]; then
    command vim $testclass
  else
    command vim "${@}"
  fi
}

So using it like this:

vim Veure::Foo::Bar::Baz

It will first look for lib/Veure/Foo/Bar/Baz.pm, and then look for t/lib/Veure/Foo/Bar/Baz.pm before finally giving up and just executing vim with whatever arguments I pass into it.

Improvements welcome.

Veure on Plack

chromatic was complaining about how tightly coupled the Veure test suite is with Catalyst. That was bugging me, too, but since I didn't yet have a compelling reason to fix this, I didn't. That was before I saw Miyagawa's Plack talk at OSCON 2010.

A feature Test::Most probably won't get

I often see this at the top of tests:

use Test::More;
if (not $ENV{TEST_AUTHOR}) {
    plan skip_all => 'Set TEST_AUTHOR to run this test';
}
plan tests => $num_tests;

In fact, I see that so often I thought about adding support for this in Test::Most.

use Test::Most tests => $num_tests, 'author';

That would have done the same thing, but Eric Wilhelm convinced me it was a bad idea. Basically, if you have author tests, put them in $module_dir/xt/author/ instead of in your regular test directory. Why? You don't have author tests run mysteriously because someone else has set that environment variable: you have to explicitly say "run my author tests". You are less likely to worry about people reporting spurious failures because they didn't have a development module installed. Why would you want to worry about those? Just use $module_dir/xt/author/ and it's a non-issue.

By further standardizing how we set up our test infrastructure, we lessen the cognitive load developers have to worry about and this is a good thing.

However, I'm willing to hear counter-arguments. Is "author" pushing too much into Test::Most or is it one of those delightful little tools you would appreciate?

Rakudo is ...

I'm not entirely sure why so many people are benchmarking Rakudo. The dev team knows that Rakudo is slow. The decision to not optimise for performance is deliberate. It's far more important to make the code correct because fast code which is incorrect isn't very helpful.

Despite our knowing what's going on, I see many people who haven't followed the Perl 6 development process (which would be just about everybody) complaining that this new language they've heard about is incredibly slow. I fear that so much talk about how sloooooow Rakudo is will start to define Rakudo in the minds of many.

We should be blogging about the neat things we can do with Rakudo. Try to create positive associations. All of this talk about performance, particularly when the dev team made it explicitly clear that they weren't focusing on performance yet, is just going to earn Rakudo a black mark.

About Ovid

user-pic Freelance Perl/Testing/Agile consultant and trainer. See http://www.allaroundtheworld.fr/ for our services. If you have a problem with Perl, we will solve it for you.