Benchmark::DKbench Perl benchmark suite now supports custom benchmarks.

Tried posting this on Reddit instead, but there seem to be some issues with code insert there, so here it is properly:
Although Benchmark::DKbench is a good overall indicator for generic CPU performance for comparing different systems (especially when it comes to Perl software), the best benchmark is always your own code. Hence, the module now lets you incorporate your own custom benchmarks. You can either have them run together with the default benchmarks, or run only your own set, just taking advantage of the framework (reports, multi-threading, monotonic precision timing, configurable repeats with averages/stdev, calculation of thread scaling etc). Here's an example where I run a couple of custom benchmarks on their own with Benchmark::DKbench:


use Benchmark::DKbench;

# A simplistic benchmark sub:
sub str_bench {
  for (1..1000) {
    my $str = join("", map { chr(97 + rand(26)) } 1..rand(15000));
    $str =~ s/a/bd/g;
    $str =~ tr/b/c/;
  }
}

my %stats = suite_run({
  include => 'custom', # Run only my custom benchmarks
  iter    => 5,        # Iterations to get an average 
  extra_bench => {
    custom_bench1 => [\&str_bench],
    # Add one more, just inline this time:
    custom_bench2 => [sub {my @a=split(//, 'x'x$_) for 1..5000}],
  }
});

This will produce a report in STDOUT and also return the results in a hash for a single-thread run. You can also run the benchmarks multi-treaded and then calculate & print the multi/single-thread scalability:


# If you want to get a count of logical cores:
my $cores = system_identity(1);

my %stats_multi = suite_run({
  include     => 'custom',
  threads     => $cores,
  iter        => 5,
  extra_bench => {
    custom_bench1 => [\&str_bench],
    custom_bench2 => [sub {my @a=split(//, 'x' x $_) for 1 .. 5000}],
  }
});

my %scal = calc_scalability(\%stats, \%stats_multi);

The report prints results per iteration and also aggregates:

Aggregates (5 iterations):
Benchmark               Avg Time (sec)      Min Time (sec)      Max Time (sec)
custom_bench1:          1.092               1.079               1.107
custom_bench2:          0.972               0.961               0.983
Overall Avg Time (sec): 2.065               2.048               2.080

Aggregates (5 iterations, 10 threads):
Benchmark               Avg Time (sec)      Min Time (sec)      Max Time (sec)
custom_bench1:          1.534               1.464               1.651
custom_bench2:          1.278               1.225               1.345
Overall Avg Time (sec): 2.812               2.689               2.965

The scalability report summarizes as well:

Multi thread Scalability:
Benchmark               Multi perf xSingle      Multi scalability %
custom_bench1:          7.12                    71
custom_bench2:          7.61                    76
----------------------------------------
DKbench summary (2 benchmarks, 5 iterations, 10 threads):
Single:             2.065s
Multi:              2.812s
Multi/Single perf:  7.36x (7.12 - 7.61)
Multi scalability:  73.6%   (71% - 76%)

The suite normally uses a scoring system which works better than times, so you can set that up by adding reference times to each benchmark, and you can also make the benchmarks return something (checksum etc) to verify results etc, see POD for more.

Leave a comment

About Dimitrios Kechagias

user-pic Computer scientist, physicist, amateur astronomer.