Benchmark your failures

Ever given any thought as to what the expense of catching exceptions with Try::Tiny or even eval might be?

Recently a colleague was having some issues with a legacy codebase that was having requests exceed their nginx proxy timeouts. We discussed increasing those timeouts, but considered that a last resort.

One block of code being executed was along the lines of


eval { do_this() };
if (! $@) {
plan_a()
} else {
plan_b()
}

which may not cause your spider sense to tingle. Note that the exception that is caught is not used in further calculations, or even reported or logged. Any tingles yet?


There are several great tools available on the CPAN to help identify slow parts of your codebase, such as the Devel::NYTProf profiler, and packages to benchmark changes such as Dumbbench. Any attempts at optimisation without first profiling your code is certainly going to be premature.

Our production servers were taking almost 40 seconds to execute the part of code that contains the above block. Profiling using same set of production data produced very unexpected results. 8.5 million calls to Carp::croak during the run and 8.5 million calls to plan_b. Hmmm... WTF?

Working through the profile output confirmed that when do_this failed, it called Carp::croak, which then generates a (short) error message relative to the caller. In isolation that would be fine; however this was happening millions times and the generated error message is ignored.

Changing the `croak` to a `die` should improve the performance as there is no need to determine the caller in our use case, but by how much? Benchmarking that hypothesis gave a 50% reduction in runtime on the same dataset. Win!

Is the problem solved? Only partially. There is still a design smell; the code needs to be redesigned to eliminate the error trapping that wraps calls to plan_b(). Handling the bulk of those cases in a more straight forward manner is a better solution. More on that at a later date!

Leave a comment

About Russell Jenkins

user-pic Random adventures of a perl hacker.