Add Structured Data to Log::Any

NB: The proposed changes have been accepted, there is now a release candidate waiting for your feedback.

A few months ago, I proposed a few changes to Log::Any with the goal of

  1. extending the API between applications and Log::Any::Adapters to allow passing hashrefs in addition to the log message
  2. allow applications to specify localized context data for Log::Any to add to each log message in that scope
  3. allow system owners to globally specify a default adapter other than Null

To get the bigger picture, you might want to revisit the original blog post and the reply by dagolden.

Based on the follow-up discussion on- and offline, I've tried to come up with a pull request that implements the proposed changes. To give it a try, clone it, and use my sample adapter to produce some output:

$ git clone https://github.com/mephinet/Log-Any.git
$ git clone https://github.com/mephinet/perl-Log-Any-Adapter-Screen.git
$ git clone https://gist.github.com/c1e48e0a6d6bbd5949483e24aeac9961.git sample
$ perl  -ILog-Any/lib/  -Iperl-Log-Any-Adapter-Screen/lib/   sample/sample.pl

For you as an application developer, if this pull requests gets applied, you can add structured data to you log lines:

$log->info("program started",
    {progname => $0, pid => $$, perl_version => $]});

Legacy adapters will receive this data flattened and appended to the log message, whereas adapters with support for structured data will receive the hashref as is, and can directly process them (e.g., send them to an ElasticSearch).

Furthermore, programs will be able to specify localized context data by adding key-value-pairs to the $log->context hash:

$log->context->{directory} = $dir;
for my $file (glob "$dir/*") {
    local $log->context->{file} = basename($file);
    $log->warn("Can't read file!") unless -r $file;
}

When a log line is written, all context data is appended to the line (for legacy adapters) or merged into the hash (for structured data adapters).

Finally, by specifying the $LOG_ANY_DEFAULT_ADAPTER environment variable, you can override the default Null adapter.

For you as a Log::Any::Adapter developer, with this pull request applied, you can specify a structured method instead of implementing a info, warn, error, etc:

sub structured {
    my ($self, $level, $category, @args) = @_;
    # ... process and log all @args
}

The method will receive level and category (as strings), followed by all arguments that were passed to the log method, unaltered, in addition with all data that has been added to the $log->context.

Please let me know what you think of these proposed changes!

2 Comments

Sounds great!

++

We are currently moving from logging to syslog to logging to/via graylog, and having structured log messages would be a huge win.

Leave a comment

About mephinet

user-pic Perl developer based in Vienna, Austria.