Wolfgang Kinkeldei

Interpreting byte code can never be faster than native code. We all know. But two benchmarks generated in different programs might tell a different story. So I thought I should share my wisdom.

Some days ago, I was experimenting with the Redis noSQL Database in order to find an alternative to file based session storage for Catalyst web applications. Redis comes bundled with a benchmark tool redis-benchmark which reveals the speed of this database. On my Macbook I get these counts when benchmarking the database with single concurrency:

$ redis-benchmark -c 1 -n 10000
[...]
====== SET ======
  10000 requests completed in 0.50 seconds
  1 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 0 milliseconds
20202.02 requests per second

====== GET ======
  10000 requests completed in 0.49 seconds
  1 parallel clients
  3 bytes payload
  keep alive: 1

100.00% <= 1 milliseconds
20408.16 requests per second
[...]

I wondered what the speed would be using Redis from Perl. I first checked the speed of Redis.pm, a very cool Perl binding. It offers lots of features, fully implements the protocol and is fast while still being a pure-perl library.

How fast could I go when sacrificing everything, Redis.pm does by simply doing network writes and reads on my own? Can I reach the C benchmark's speed? Well, Perl is even faster -- and I also use JSON::XS to encode a hash :-) At least my benchmark is telling me...

Benchmark: timing 10000 iterations of get_native, get_redis, set_native, set_redis...
get_native:  0 wallclock secs ( 0.23 usr +  0.13 sys =  0.36 CPU) @ 27777.78/s (n=10000)
            (warning: too few iterations for a reliable count)
 get_redis:  2 wallclock secs ( 0.95 usr +  0.20 sys =  1.15 CPU) @ 8695.65/s (n=10000)
set_native:  1 wallclock secs ( 0.20 usr +  0.14 sys =  0.34 CPU) @ 29411.76/s (n=10000)
            (warning: too few iterations for a reliable count)
 set_redis:  1 wallclock secs ( 0.82 usr +  0.21 sys =  1.03 CPU) @ 9708.74/s (n=10000)
              Rate  get_redis  set_redis get_native set_native
get_redis   8696/s         --       -10%       -69%       -70%
set_redis   9709/s        12%         --       -65%       -67%
get_native 27778/s       219%       186%         --        -6%
set_native 29412/s       238%       203%         6%         --

The full benchmark, also comparing JSON::XS versus YAML::XS and pure writes without any serialization can be found here: https://gist.github.com/1697936