Lazy == Cache ?
While I was working on some Moose code this past week, I was struck (again) by how the lazy property of attributes functions almost like a cache. In fact, often when working in Moose I find that when I want something to be cached, I just make it lazy and call it a day.
Let me illustrate what I’m talking about with some code. Now, the real process of putting this code together involved a lot of blind alleys and false starts, but I’m going to present it like I came up with the final product smoothly and without interruption. I do this for two reasons: one, it makes for less frustrating reading, and, two, it makes me look smarter. ;->
So I’m creating a little command line utility to do some moderately complex data loading. I decide to use MooseX::App::Cmd for the basic structure—this is an excellent module that I’m using more and more these days. Definitely check it out.
So, I’m going to need to some support classes. Since this utility is going to need to know things about data structures, indices, foreign keys, etc, I’ll need a class to manage the database stuff.
use Company::Moose;
class Rent::Data::DB
{
use MooseX::Types::Moose qw< :all >;
has name => (isa => Str, is => 'ro', required => 1);
has user => (isa => Str, is => 'ro');
has Dbh => (isa => 'DBI::db', is => 'ro', lazy => 1, builder => '_connect_dbh');
}
Pretty basic:
- The
Company::Moosemodule is a “policy module” that gathers up all the common Moose stuff we want for our company. See the second example in the synopsis of Method::Signatures::Modifiers for what it basically looks like. - The class name starts with
Rentbecause that’s what $work is. ThenData::DBbecause is it’s a class that deals with our database, but specifically our particular data structures within that DB. (In retrospect, I probably should have tried to work “schema” in there somewhere. Ah, well.) - It has 3 basic attributes: database name, user to log in as (not required because there is a default user), and the actual DBI database handle.
- Not shown would be the constructor,
connect, and the builder for theDbhattribute,_connect_dbh, which utilize a bunch of company-specific connection stuff.
Rent::Data::DB for the primary key columns of a table, dependencies and reverse depdencies, and so forth.
Except ... maybe my “DB” class should really be a container. Just like a real database, it should contain a bunch of tables. Okay, fine, now I need a table class:
use Company::Moose;
class Rent::Data::DB::Table
{
use MooseX::Types::Moose qw< :all >;
has name => (isa => Str, is => 'ro', required => 1);
has db => (isa => 'Rent::Data::DB', is => 'ro', required => 1, weak_ref => 1, handles => ['run_query']);
}
class Rent::Data::DB
{
use MooseX::Types::Moose qw< :all >;
has name => (isa => Str, is => 'ro', required => 1);
has user => (isa => Str, is => 'ro');
has Dbh => (isa => 'DBI::db', is => 'ro', lazy => 1, builder => '_connect_dbh');
has _tables => (isa => HashRef, is => 'ro', lazy => 1, default => sub { {} });
# sort of a pseudo-attribute
method table (Str $tablename)
{
unless (exists $self->_tables->{$tablename})
{
$self->_tables->{$tablename} = Rent::Data::DB::Table->new(name => $tablename, db => $self);
}
return $self->_tables->{$tablename};
}
}
Wow, that was pretty simple. My “table” class has a name, and a reference to its parent DB. Since the parent will also have a reference to the table, I make the
db attribute a weak reference. Then I make a HashRef attribute of Rent::Data::DB which will hold all the table objects. It’s lazy, of course, starts off as an empty hashref (tablename => table_obj), and then I create a table method which will create table objects on demand, and return the cached results.
This is pretty cool, right? But it gets even cooler. How about a way to get primary key columns?
use Company::Moose;
class Rent::Data::DB::Table
{
use MooseX::Types::Moose qw< :all >;
has name => (isa => Str, is => 'ro', required => 1);
has db => (isa => 'Rent::Data::DB', is => 'ro', required => 1, weak_ref => 1, handles => ['run_query']);
has pk_cols => (isa => ArrayRef, is => 'ro', lazy => 1, builder => '_get_pk_cols');
method _get_pk_cols
{
my $query = q{
select cc.column_name
from all_constraints c, all_cons_columns cc
where c.table_name = ?
and c.constraint_type = 'P'
and c.owner = cc.owner
and c.constraint_name = cc.constraint_name
};
return [ map { lc } $self->run_query($query, uc $self->name) ];
}
}
class Rent::Data::DB
{
use MooseX::Types::Moose qw< :all >;
has name => (isa => Str, is => 'ro', required => 1);
has user => (isa => Str, is => 'ro');
has Dbh => (isa => 'DBI::db', is => 'ro', lazy => 1, builder => '_connect_dbh');
has _tables => (isa => HashRef, is => 'ro', lazy => 1, default => sub { {} });
# sort of a pseudo-attribute
method table (Str $tablename)
{
unless (exists $self->_tables->{$tablename})
{
$self->_tables->{$tablename} = Rent::Data::DB::Table->new(name => $tablename, db => $self);
}
return $self->_tables->{$tablename};
}
# we can handle all methods that a Rent::Data::DB::Table can handle
# args will be the same except the table name is the first arg
foreach my $method (Rent::Data::DB::Table->meta->get_method_list)
{
next if $method eq uc $method; # special methods like BUILD and DESTROY
next if $method =~ /^_/; # private methods
next if $method eq 'new' or $method eq 'meta'; # the two special methods that are lc
next if grep { $_ eq $method } qw< name db run_query >; # not special, but not required here either
__PACKAGE__->meta->add_method($method, sub
{
my $self = shift;
my $tablename = shift;
my $table = $self->table($tablename);
return $table->$method(@_);
});
}
}
Easy peasy. I just add a lazy attribute called
pk_cols and give it an appropriate builder (revealing my secret shame: yes, I’m using Oracle (no, it’s not by choice)). Now, every time I call pk_cols on a table object, I get the appropriate columns, and because it’s an attribute (instead of a method) it’s stored (essentially cached). The bit of trickery in the DB class just allows me a little shortcut:
# instead of this:
my $cols = $db->table($table)->pk_cols;
# I can just do this:
my $cols = $db->pk_cols($table);
use Company::Moose;
class Rent::Data::DB::Table
{
use MooseX::Types::Moose qw< :all >;
has name => (isa => Str, is => 'ro', required => 1);
has db => (isa => 'Rent::Data::DB', is => 'ro', required => 1, weak_ref => 1, handles => ['run_query']);
has pk_cols => (isa => ArrayRef, is => 'ro', lazy => 1, builder => '_get_pk_cols');
has indexes => (isa => HashRef, is => 'ro', lazy => 1, builder => '_get_indexes', traits => ['Hash'],
handles =>
{
is_indexed => 'exists',
},
);
method _get_pk_cols
{
my $query = q{
select cc.column_name
from all_constraints c, all_cons_columns cc
where c.table_name = ?
and c.constraint_type = 'P'
and c.owner = cc.owner
and c.constraint_name = cc.constraint_name
};
return [ map { lc } $self->run_query($query, uc $self->name) ];
}
method _get_indexes
{
my $query = q{
select distinct aic.column_name
from all_indexes ai, all_ind_columns aic
where ai.table_name = ?
and ai.index_name = aic.index_name
and aic.column_name not like '%$'
and aic.column_position = 1
};
return { map { lc $_ => 1 } $self->run_query($query, uc $self->name) };
}
}
class Rent::Data::DB
{
use MooseX::Types::Moose qw< :all >;
has name => (isa => Str, is => 'ro', required => 1);
has user => (isa => Str, is => 'ro');
has Dbh => (isa => 'DBI::db', is => 'ro', lazy => 1, builder => '_connect_dbh');
has _tables => (isa => HashRef, is => 'ro', lazy => 1, default => sub { {} });
# sort of a pseudo-attribute
method table (Str $tablename)
{
unless (exists $self->_tables->{$tablename})
{
$self->_tables->{$tablename} = Rent::Data::DB::Table->new(name => $tablename, db => $self);
}
return $self->_tables->{$tablename};
}
# we can handle all methods that a Rent::Data::DB::Table can handle
# args will be the same except the table name is the first arg
foreach my $method (Rent::Data::DB::Table->meta->get_method_list)
{
next if $method eq uc $method; # special methods like BUILD and DESTROY
next if $method =~ /^_/; # private methods
next if $method eq 'new' or $method eq 'meta'; # the two special methods that are lc
next if grep { $_ eq $method } qw< name db run_query >; # not special, but not required here either
__PACKAGE__->meta->add_method($method, sub
{
my $self = shift;
my $tablename = shift;
my $table = $self->table($tablename);
return $table->$method(@_);
});
}
}
now this just works too:
my $indices = $db->indexes($table);
See? Instant caching. Now, every attribute I add to the table object from here on out gives me an on-demand, insta-cached property of tables for my Data::DB. It’s pretty damn cool, if I do say so myself.
Note that the real code includes a bit more stuff: more comments, more infrastructure, and my possibly-too-clever run_query method, which includes error checking and debugging, and makes snap decisions about whether you want a scalar, an array, or an array (or arrayref) of hashrefs, based on caller context and number of columns being requested. But this should give you the flavor of what I’m talking about for this post.
Hopefully this will inspire you to utilize Moose’s lazy property for some caching of your own. All we need now are Moose properties for impatient and hubristic and we’ll be all set.
14 years in California, 25 years in Perl, 34 years in computers, 55 years in bare feet.
chromatic posted something about this recently too...
http://www.modernperlbooks.com/mt/2012/02/the-memoization-of-lazy-attributes.html
In one of my current projects virtually everything happens in _builder_* subs.
> chromatic posted something about this recently too...
Ah, I must've missed that one. Should've known chromatic would beat me to the punch. :) Oh, well, hopefully my code still has something useful to add.