Redis
and IO-Socket-Timeout
.
If you're interested by some or all of them, let me know!
]]>In Perl, exceptions are a well known and widely used mechanism. It is an old
feature that has been enhanced over time. At the basic level, exceptions are
triggered by the keyword die
. Exceptions were initially used as a way to stop
the execution of a program in case of a fatal error. The too famous line:
open my $fh, $file or die "failed to open '$file', error: $!";
is a good example.
The original way to catch exceptions in Perl has a somewhat strange syntax,
it's based on the eval
keyword and the special variable $@
:
eval { code_that_may_die(); 1; }
or say "exception has been caught: $@"
Nowadays, exceptions are usually thrown using croak
and friends, from the
Carp
module. It allows for a much better flexibility about where the exception
seems to originate, and how to display the stack trace, if any.
Catching exceptions with eval
is also supersed by try/catch mechanisms. The
most used one is via the [Try::Tiny
][try-tiny] module by Yuval Kogman and Jesse Luehrs,
and goes like this:
try {
croak "exception";
} catch {
warn "caught error: $_";
};
The good thing about die
(or croak
), is that it's very easy to use, when
given a string. It's perfect for using in scripts, or moderately big
projects. However, for more features, or extensive usage of exceptions, then
it's better to throw objects instead of strings, like this:
open $file or die MyExceptions::IO::File->new(
filename => $file,
error => $!
);
For this snippet of code to work, the MyExceptions::IO::File
class has to be
declared, its fields as well, and the it should probably inherit from
MyExceptions::IO
. So it requires some amount of work.
Some modules have been created - long time ago - to automate or help with
declaring exception classes. The most well known one is [Exception::Class
][exception-class], by
Dave Rolsky. For instance, here is how to declare two exceptions matching with
previous example:
package MyExceptions;
use Exception::Class (
'MyException::IO',
'MyException::IO::File' => {
isa => 'MyException::IO',
fields => [ 'filename' ],
},
);
And then, here is the code to make use of that and throw an exception when failing to open a file:
use MyExceptions;
open $file or MyException::IO::File->throw(
filename => $file,
error => $!
);
When using objects as exceptions, a set of features becomes available, thanks to Object Oriented Programming. Inheritance, attributes and introspection are some of them. However the most visible and used feature is about catching such exceptions:
use MyException;
try {
open $file or MyException::IO::File->throw(
filename => $file,
error => $!
);
} catch {
my $exception = $_;
if ($exception->isa(MyException::IO)) {
# we know how to handle these
} else {
$exception->rethrow
}
};
As you can see, it's easy to introspect an exception if it's an object. In this
case we use the isa
keyword to know if the exception is or inherits from a
given class name.
As we saw in the previous chapter, Perl allows exceptions being whatever you like (string, objects, but actually numbers, structures, etc, work as well).
Usually, when starting a project, the author decides whether to use simple strings or objects with a class hierarchy. With very big projects, it is sometimes not possible to impose one kind of exceptions. This may be due to legacy code, a subproject that was included, or the wish to give people freedom about what they want to use depending on the context.
In these cases, the code may have to handle exceptions of two kinds: strings and objects. This can be done via this kind of code:
use MyException;
use Scalar::Util qw(blessed);
try {
# ... code that may die
} catch {
my $exception = $_;
if (blessed $exception) {
# exception is an object
# ...
} else {
# exception is a normal string
# ...
}
};
The previous code snippet suffers from increased complexity due to the additional checks and two different codepaths for handling potential errors. This is clearly both suboptimal and error prone.
Another issue is that some code may consider that the exception it is catching is of one type, whereas it could be of an other type, especially because of the action-at-distance nature of the exception. Consider this function:
sub do_stuff {
try {
# ... code that can only throw objects exceptions
} catch {
my $exception = $_;
# exception is always an object
if ($exception->isa(...)) {
# ...
}
};
}
This code assumes that the exception will always be an object. However,
let's consider this: in following example, the function do_stuff
is called
(its original code is unchanged), but before doing so, the special signal
handler for __DIE__
is changed.
$SIG{__DIE__} = sub { die "FATAL: $_[0]" };
do_stuff();
The first line of the example is being called when an exception is raised, and
will be executed instead of propagating the exception. What this code does is
prepending FATAL:
to it, then propagate the exception again by using die
.
Alas, it is doing so in a naive way, by forcing the exception (in $_[0]
) to be
evaluated as a string. So when the exception is then re-thrown, it is now a
string ! and Boom, the ->isa
call in do_stuff
won't work.
The worst thing about this kind of issue is that it doesn't appear at compile time, nor at execution time, but at exception time, which is the worst time...
So at that point, most developers will choose the following strategy. Use object exceptions for their code, but guard against receiving string exceptions, and also make their object exceptions nicely degrade into strings, by using stringification overloading. That means that if an object exception is managed by a handler that threats it as a string, the exception will transform itself into a string, and try to present some meaningful aspect of itself.
The issue is that handling exception is now back to square one, having to deal with strings, trying to parse it looking for meaningful information to hopefully make a good decision.
What if, instead of taking an object exception and downgrading it to a string while keeping as much information as possible, one starts from a string, and enhance it until it looks like an object, without being one ? That way we would have the best of both worlds
This is what Exception::Stringy
tries to achieve.
A perfect exception would have these features:
This set of features is not big, but it's probably enough for a start. Let's see how we can implement them in a simple string. We're going to use an exception with these attributes:
filename
Let's start with the first feature: be a string, containing an error message. That's easy:
"permission denied"
Being an instance of a class is usually done in Perl by using bless
on a
ScalarRef. But we don't want the eception to be an object. What bless
does -
and what it ultimately means to "be an instance of a class", is just attaching
a label to a value. Let's do that, by having a label as a substring in our
exception. For instance:
"[MyException::IO]permission denied"
We could add a magic mark or have a more complex label syntax to make sure it's a legit label.
To know what the class of a given exception is, we just need to extract the label, for instance with a regex.
Inheritance is easy, it only requires that standard Perl classes be created to map the exception labels, and then Perl usual inheritance can be used.
So, following our example, we need two packages, MyException
and
MyException::IO
, and @MyException::IO::ISA
set to ['MyException']
. This can
be made automatically at exception declaration time.
For simplicity, Exception::Stringy
only handles simple field values, that is
strings and numbers basically. To put fields into our string, we need to be
able to identify them, so for instance with a separator between the different
fields, and an other one between a field name and its value. Like this:
"[MyException::IO|filename:/tmp/file|]permission denied"
And if the field name or value contains one of the separators ( [
, |
, :
or ]
), let's encode them in base64, and mark it as such.
So, by now, we have fleshed out a string with useful data, which is properly parseable, and can be described. Let's add methods to the data now.
Given an exception, it is mandatory to be able to introspect and modify it, namely be able to:
In an ideal world, we would want methods, that we can call on our exception instances. However because our exceptions are regular strings, we can't do this:
$exception->message();
Usually, this way of calling a method (the arrow notation) works only if $exception is a blessed reference (that is, an object). However, there are other cases in which we can use the arrow notation, and have it work in a similar way. One of it is this one:
$exception->$message();
If $message is a variable that contains a reference on a subroutine, then the previous line will translate into:
$message->($exception);
And it works whatever the type of $exception
, like in our case, a string. So,
Exception::Stringy
creates the needed subroutine references for the user and
allow such arrow notation, which is very similar to the OO method invocation. I
call these pseudo methods.
However, to avoid clobbering an existing variable, the pseudo methods need to
have names that are unlikely to be already used in the target package. It's
even better if there is an option to add a prefix to these pseudo-methods.
Once again, Exception::Stringy
provides these features. The default pseudo
method names are :
$exception->$xthrow()
$exception->$xrethrow()
$exception->$xraise()
$exception->$xclass()
$exception->$xisa()
$exception->$xfields()
$exception->$xfield()
$exception->$xmessage()
$exception->$xerror()
Finally, once we have created the exception, let's throw it. The first think to
do is to implement a throw
or `raise class method on all the exception class,
so that we can do
MyException->throw(...)
That will basically craft a new exception string, with all the properties
encoded in it, and call die
or croak
on it.
We can also use a pseudo method on an existing exception to (re)throw it:
$exception->$xthrow();
Below is the synopsis of the Exceptions::Stringy
module. It's basically a
wrap up of what has been explained above. The exceptions definition is heavily
inspired from Exception::Class
.
use Exception::Stringy;
Exception::Stringy->declare_exceptions(
'MyException',
'YetAnotherException' => {
isa => 'AnotherException',
},
'ExceptionWithFields' => {
isa => 'YetAnotherException',
fields => [ 'grandiosity', 'quixotic' ],
throw_alias => 'throw_fields',
},
);
### with Try::Tiny
use Try::Tiny;
try {
# throw an exception
MyException->throw('I feel funny.');
# or use an alias
throw_fields 'Error message', grandiosity => 1;
# or with fields
ExceptionWithFields->throw('I feel funny.',
quixotic => 1,
grandiosity => 2);
# you can build exception step by step
my $e = ExceptionWithFields->new("The error message");
$e->$xfield(quixotic => "some_value");
$e->$xthrow();
} catch {
if ( $_->$xisa('Exception::Stringy') ) {
warn $_->$xerror, "\n";
}
if ( $_->$xisa('ExceptionWithFields') ) {
if ( $_->$xfield('quixotic') ) {
handle_quixotic_exception();
} else {
handle_non_quixotic_exception();
}
} else {
$_->$xrethrow;
}
};
### without Try::Tiny
eval {
# ...
MyException->throw('I feel funny.');
1;
} or do {
my $e = $@;
# .. same as above with $e instead of $_
}
This was an in-depth look at why and how to build up a resilient and non-intrusive exception mecanism. I hope to have demonstrated one aspect of the extreme flexibility of Perl.
Feel free to use Exception::Stringy
, it is being used in production code for
some time now. Feedback welcome !
my $content = do{local(@ARGV,$/)=$filename;<>};
I don't think it's useful to benchmark it :)
]]>This is going to be a short post. I'm the new maintainer of Redis.pm, the most used Redis Perl client. Pedro Melo was the previous maintainer, but due to Real Life, he is unable to continue. I'd like to thank him for all his efforts so far in maintaining and improving this module. I hope I'll be able to achieve the same level of quality. Pedro will actually stay around for a while, watching over my shoulder and giving his opinions about stuff, to allow for a smooth transition.
We've used this maintainership change to improve the tools we use around this project. So we've moved the code to the github's PerlRedis organization (notice the cool logo), and I've performed quite a few code cleanups and housekeeping.
But the big thing is the creation of a mailing list, that aims at gathering forces around Redis support in Perl. It's located here, and hosted by the good folks at ShadowCat Systems Limited (thank you guys).
It is not limited to the Redis.pm module: any Perl related Redis topic is welcome, including other Perl clients. So if you have any interest in Perl and Redis, feel free to subscribe !
dams.
]]>If not, in a nutshell, p5-mop is an attempt to implement a subset of Moose into the core of Perl. Moose provides a Meta Object Protocol (MOP) to Perl. So does p5-mop, however p5-mop is implemented in a way that it can be properly included in the Perl core.
Keep in mind that p5-mop goal is to implement a subset of Moose. As Stevan Little says:
We are not putting "Moose into the core" because Moose is too opinionated, instead we want to put a minimal and less opinionated MOP in the core that is capable of hosting something like Moose
As far as I understood, after a first attempt that failed, Stevan Little restarted the p5-mop implementation: the so-called p5-mop-redux github project, using Devel::Declare, ( then Parse::Keyword ), so that he can experiment and release often, while keeping the implementation core-friendly. Once he's happy with the features and all, he'll make sure it finds its way to the core. A small team (Stevan Little, Jesse Luehrs, and other contributors) is actively developping p5-mop, and Stevan is regularly blogging about it.
If you want more details about the failing first attempt, there is a bunch of backlog and mailing lists archive to read. However, here is how Stevan would summarize it:
We started the first prototype, not remembering the old adage of "write the first one to throw away" and I got sentimentally attached to my choice of design approach. This new approach (p5-mop-redux) was purposfully built with a firm commitment to keeping it as simple as possible, therefore making it simpler to hack on. Also, instead of making the MOP I always wanted, I approached as building the mop people actually needed (one that worked well with existing perl classes, etc)
Few months ago, when p5-mop-redux was announced, I tried to give it a go. And you should too ! Because it's easy.
It's important to have at least a vague idea of where p5-mop stands at, because this project is shaping a big part of Perl's future. IMHO, there will be a before and an after having a MOP in core. And it is being designed and tested right now. So as Perl users, it's our chance to have a look at it, test it, and give our feedback.
Do we like the syntax ? Is it powerful enough? What did we prefer more/less in Moose ? etc. In few months, things will be decided and it'll only be a matter of time and implementation details. Now is the most exciting time to participate in the project. You don't need to hack on it, just try it out, and provide feedback.
p5-mop is very easy to install:
curl -L http://cpanmin.us | perl - App::cpanminus
cpanm --dev twigils
p5-mop-redux
project. Otherwise you can get a zip here.cpanm .
from within the p5-mop-redux directory.Here is the classical point example from the p5-mop test suite
use mop;
class Point {
has $!x is ro = 0;
has $!y is ro = 0;
method set_x ($x) {
$!x = $x;
}
method set_y ($y) {
$!y = $y;
}
method clear {
($!x, $!y) = (0, 0);
}
method pack {
+{ x => $self->x, y => $self->y }
}
}
# ... subclass it ...
class Point3D extends Point {
has $!z is ro = 0;
method set_z ($z) {
$!z = $z;
}
method pack {
my $data = $self->next::method;
$data->{z} = $!z;
$data;
}
}
This examples shows how straightforward it is to declare a class and a subclass. The syntax is very friendly and similar to what you may find in other languages.
class
declares a class, with proper scoping. method
is used to define
methods, so no sub
there. The distinction is important, because in methods,
additional variables will be automatically available:
$self
will be available directly, no need to shift @_
.$self->accessors
.Functions defined with the regular sub
keyword won't have all these features,
and that's for good: it makes the difference between function and method
more explicit.
has
declares an attribute. Attribute names are twigils. Borrowed from Perl6,
and implemented by Florian Ragwitz in its
twigils project on github, twigils are
useful to differenciate standard variables from attributes variables:
class Foo {
has $!stuff;
method do_stuff ($stuff) {
$!stuff = $stuff;
}
}
As you can see, it's important to be able to differenciate stuff
(the
variable) and stuff
(the attribute).
The added benefit of attributes variables is that one doesn't need to contantly
use $self
. A good proportion of the code in a class is about attributes.
Being able to use them directly is great.
Other notes worth mentiong:
BUILD
method, as with Moose.extend
-ing it.SUPER
,
but $self->next::method
.Foo
declared in the package Bar
will be defined as Bar::Foo
.When declaring an attribute name, you can add is
, which is followed by a list of
traits:
has $!bar is ro, lazy = $_->foo + 2;
ro
/ rw
means it's read-only / read-writelazy
means the attribute constructor we'll be called only when the
attribute is being usedweak_ref
enables an attribute to be a weak referencehas $!foo = 'default value';
which is actually
has $!foo = sub { 'default value' };
So, there is no default value, only builders. That means that has $!foo = {};
will work as expected ( creating a new hashref each time ).
You can reference the current instance in the attribute builder by using $_
:
has $!foo = $_->_init_foo;
There has been some comments about using =
instead of //
or ||
or
default
, but this syntax is used in a lot of other programing language, and
considered somehow the default (ha-ha) syntax. I think it's worth sticking with
=
for an easier learning curve for newcomers.
UPDATE: Similarly to attributes, classes and methods can have traits. I won't go in details to keep this post short, but you can make a class abstract, change the default behaviour of all its attributes, make it work better with Moose, etc. Currently there is only one method trait to allow for operator overloading, but additional ones may appear shortly.
When calling a method, the parameters are as usual available in @_
. However
you can also declare these parameters in the method signature:
method foo ($arg1, $arg2=10) {
say $arg1;
}
Using =
you can specify a default value. In the method body, these parameters
will be available directly.
Types are not yet core to the p5-mop, and the team is questioning this idea. The concensus is currently that types should not be part of the mop, to keep it simple and flexible. You ought to be able to choose what type system you want to use. I'm particularly happy about this decision. Perl is so versatile and flexible that it can be used (and bent to be used) in numerous environment and configuration. Sometimes you need robustness and high level powerful features, and it's great to use a powerful typing system like Moose's one. Sometimes (most of the time? ) Type::Tiny (before that I used Params::Validate) is good enough and gives you faster processing. Sometimes you don't want any type checking.
Because the attribute builder is already implemented using =
, what about
clearer and predicate?
# clearer
method clear_foo { undef $!foo }
# predicate
method has_foo { defined $!foo }
That was pretty easy, right? Predicates and clearers have been introduced in
Moose because writing them ourselves would require to access the underlying
HashRef behind an instance (e.g. sub predicate { exists $self->{$attr_name}}
)
and that's very bad. To work around that, Moose has to generate that kind of
code and provide a way to enable it or not. Hence the predicate
and clearer
options. So you see that they exists mostly because of the implementation.
In p5-mop, thanks to the twigils, there is no issue in writing predicates and cleare ourselves.
But I hear you say "Wait, these are no clearer nor predicate ! They are not testing the existence of the attributes, but their define-ness!" You're right, but read on!
In Moose there is a difference between an attribute being unset, and an attribute being undef. In p5-mop, there is no such distinction. Technically, it would be very difficult to implemente that distinction, because an attribute variable is declared even if the attribute has not been set yet.
In Moose, because objects are stored in blessed hashes, an attribute can either be:
That's probably too many cases... Getting rid of one of them looks sane to me.
After all, we got this "not set" state only because objects are stored in HashRef, so it looks like it's an implementation detail that made its way into becoming a concept on its own, which is rarely a good thing.
Plus, in standard Perl programming, if an optional argument is not passed to a function, it's not "non-existent", it's undef:
foo();
sub foo {
my ($arg) = @_; # $arg is undef
}
So it makes sense to have a similar behavior in p5-mop - that is, an attribute that is not set is undef.
Roles definition syntax is quite similar to defining a class.
role Bar {
has $!additional_attr = 42;
method more_feature { say $!additional_attr }
}
They are consumed right in the class declaration line:
class Foo with Bar, Baz {
# ...
}
Going meta is not difficult either but I won't describe it here, as I just want to showcase default OO programming syntax. On that note, it looks like Stevan will make classes immutable by default, unless specified. I think that this is a good idea (how many time have you written make_immutable ?).
Method modifiers are not yet implemented, but they won't be difficult to
implement. Actually, here is an example of how to implement method modifiers
using p5-mop very own meta. It implements around
:
sub modifier {
if ($_[0]->isa('mop::method')) {
my $method = shift;
my $type = shift;
my $meta = $method->associated_meta;
if ($meta->isa('mop::role')) {
if ( $type eq 'around' ) {
$meta->bind('after:COMPOSE' => sub {
my ($self, $other) = @_;
if ($other->has_method( $method->name )) {
my $old_method = $other->remove_method( $method->name );
$other->add_method(
$other->method_class->new(
name => $method->name,
body => sub {
local ${^NEXT} = $old_method->body;
my $self = shift;
$method->execute( $self, [ @_ ] );
}
)
);
}
});
} elsif ( $type eq 'before' ) {
die "before not yet supported";
} elsif ( $type eq 'after' ) {
die "after not yet supported";
} else {
die "I have no idea what to do with $type";
}
} elsif ($meta->isa('mop::class')) {
die "modifiers on classes not yet supported";
}
}
}
It is supposed to be used like this:
method my_method is modifier('around') ($arg) {
$arg % 2 and return $self->${^NEXT}(@_);
die "foo";
}
I would like to see method modifiers in p5-mop. As per Stevan Little and Jesse
Luehrs, it may be that these won't be part of the mop, but in a plugin or
extension. I'm not to sure about that, for me method modifier is really linked
to OO programmning. I prefer using around
than fiddling with
$self->next::method
or ${^NEXT}
.
Here are some syntax proposals I've gathered on IRC and blog comments regarding what could be method modifiers in p5-mop:
around foo { }
method foo is around { ... }
method foo is modifier(around) { ... }
These special variables are pointing to the current instance (useful when
you're not in a method - otherwise $self
is available), and the next method
in the calling chain. It's OK to have such variables, but their horrible name
makes it difficult to remember and use.
Can't we have yet an other type of twigils for these variables ? so that we can
write $^NEXT
and $^SELF
.
Just an idea, but maybe we could have $!public_attribute
and
$.private_attribute
. Or is it the other way around ?
is
? we already have has
!This one thing is bothering me a lot: why do we have to use the word is
when
declaring an attribute? The attribute declaration starts with has
. So with
is
, that makes it two verbs for one line of code. For me it's too much.
in Moo* modules, the is
was just one property. We had default
, lazy
,
etc. Now, is
is just a seperator between the name and the 'traits'. In my
opinion, it's redundant.
Also, among the new keywords added by p5-mop, we have only nouns (class
,
role
, method
). Only one verb, has
.
The counter argument on this is that this syntax is inspired by Perl6:
class Point is rw {
has ($.x, $.y);
method gist { "Point a x=$.x y=$.y" }
}
So, "blame Larry" ? :)
p5-mop doesn't use @ISA for inheritance, so use base 'Exporter'
won't work.
You have to do use Exporter 'import'
. That is somewhat disturbing because
most Perl developers (I think) implement functions and variables exporting by inheriting from Exporter (that's also what the documentation of Exporter recommends).
You could argue that one should code clean classes (that don't export anything, and clean modules (that export stuff but don't do OO). Mixing OO in a class with methods and exportable subs looks a bit un-orthodox. But that's what we do all day long and it is almost part of the Perl culture now. Think about all the modules that provides 2 APIs, a functional one and an OO one. All in the same namespace. So, somehow, being able to easily export subs is needed.
However, as per Jesse Luehrs and Stevan Little, they don't think a MOP
implementation should be in charge of implementing an Exporter module, and I
can totally agree with this. So it looks like the solution will be a method
trait, like exportable
:
sub foo is exportable { ... }
But that is not yet implemented.
p5-mop is not using the standard scheme where an object is simply a blessed
structure (usually a HashRef
). Instead, it's using InsideOut objects, where
all you get as an object is some kind of identification number (usually a
simple reference), which is used internally to retrieve the object properties,
only accessible from within the class.
This way of doing may seem odd at first: if I recall correctly, there a time
where InsideOut objects were trendy, especially using Class::Std
. But that
didn't last long, when Moose and its follow ups came back to using regular
blessed structured objects.
The important thing to keep in mind is that it doesn't matter too much. Using inside out objects is not a big deal because p5-mop provides so much power to interact and introspect with the OO concepts that it's not a problem at all that the attributes are not in a blessed HashRef.
However, a lot of third-party modules assume that your objects are blessed HashRef. So when switching to p5-mop, a whole little ecosystem will need to be rewritten.
UPDATE: ilmari pointed out in the comments that there is a class trait called repr
that makes it possible to change the way an instance is implemented. You can specify if an object should be a reference on a scalar, array, hash, glob, or even a reference on a provided CodeRef. This makes p5-mop objects much more compatible with the OO ecosystem.
Now, it's your turn to try it out, make up your mind, try to port an module or write on from scratch using p5-mop, and give your feedback. To do that, go to the IRC channel #p5-mop on the irc.perl.org server, say hi, and explain what you tried, what went well and what didn't, and how you feel about the syntax and concepts.
Also, spread the word by writing about your experience with p5-mop, for instance on blogs.perl.org.
Lastly, don't hesitate to participate in the comments below :) Especially if you don't agree with my remarks above.
This article has been written by Damien Krotkine, but these people helped proof-reading it:
Just a quick note to mention that following Mike Doherty's bug report, I've released a new version of MooX::LvalueAttribute.
This release (version 0.12) allows you to use MooX::LvalueAttribute in a Moo::Role, like this;
{
package MyRole;
use Moo::Role;
use MooX::LvalueAttribute;
}
{
package MyApp;
use Moo;
with ('MyRole');
has name => ( is => 'rw',
lvalue => 1,
);
}
my $object = MyApp->new();
$object->name = 'Joe';
So now it's easier to specify which classes will have lvalue attributes and
which one won't. Until now I avoided adding a flag to globally enable lvalue
attributes across all Moo classes (without having to say lvalue => 1
). Maybe
that's something some of you would like ?
Anyway, that's all folks! Nothing revolutionary, but I've been told we should talk more about what we do, so that's what I'm doing.
For more detail about Moox::LvalueAttribute, see my original post
]]>You know how some coders lack imagination regarding variable names
has $fetcher = ...... 200 lines later ...
method foo {
my ($fetchers, $fetching_data, $data_fetched, $fletch) = ...;
$fetchers->($fletch, $fetcher->($fetching_data));
}
The example is stupid, but you see how it can be difficult to pinpoint the attribute. Which is IMO a very important feature of an OO syntax. Also, companies using Perl will then enforce "coding styles" that imply prefixing all attributes with "attr_" or something... Any thoughts ?
thanks to @yenzie for the picture :P
]]> Bloom filtersBloom filters are statistical data structures. The most common use of them is to consider them as buckets. In one bucket, you add elements. Once you've added a bunch of elements, it's ready to be used.
You use it by presenting it yet an other element, and it'll be able to say almost always if the element is already in the bucket or not.
More precisely, when asking the question "is this element in the filter ?", if it answers no, then you are sure that it's not in there. If it answers yes, then there is a high probability that it's there.
So basically, you never have false negatives, but you can get a few false positives. The good thing is that depending on the space you allocate to the filter, and the number of elements it contains, you know what will be the probability of having false positives.
The huge benefit is that a bloom filter is very small, compared to a hash table.
At work, I replaced a heavy Redis instance ( using 60g of RAM) that was used primarily as a huge hash table, by a couple of bloom filters ( using 2g ). For that I used bloomd, from Armon Dadgar. It's light, fast, has enough features, and the code looks sane.
All I needed was a Perl client to connect to it.
So I wrote Bloomd::Client. It is a light client that connects to bloomd using a regular INET socket, and speaks the simple ASCII protocol (very similar to Redis' one) that bloomd implements.
use Bloomd::Client;
my $b = Bloomd::Client->new;
my $filter = 'test_filter';
$b->create($filter);
my $hash_ref = $b->info($filter);
$b->set($filter, 'u1');
if ($b->check($filter, 'u1')) {
say "it exists!"
}
When you use bloomd it usually means that you are in a high availibility environment, where you can't get stuck waiting on a socket, just because something went wrong. So Bloomd::Client implements non-blocking timeouts on the socket. It'll die if bloomd didn't answer fast enough or if something broke. That allows you to incorporate the bloomd connection in a retry strategy to try again later, or fallback to another server...
To implement such a strategy, I recommend using Action::Retry. There is a blog post about it here :)
dams.
]]>