Asynchronous Task Distribution with AnyEvent and ZeroMQ

Some months ago i wrote how to (ab)use your database as a messagequeue in order to distribute tasks among worker-processes.

"From your code, it also looks like having more than one demon will put you at risk of processing the same jobs more than once." (Jerome Eteve)

"Abuse" is the right word. The moment you hit more than three concurrent jobs, you will see a nasty slowdown ..." (rob.kinyon)

The comments made pretty clear that this was not a go…

bulk inserting tens of thousands of rows with DBIx::Class

For some crazy reason my $project had a denormalized database layout. I had a table 'job' with a column 'data' which had a serialized ArrayRef of HashRefs as value.

But performance turned out to be really bad, I did not take any premature measu…

Relationships with optional foreign key values (DBIx::Class)

This entry demonstrates how to implement a relationship with optional foreign key values using DBIx::Class.

Imagine two related tables 'human' and 'cat': A cat belongs to a human but when the human dies the cat can continue to roam freely.

package MyApp::Schema::Result::Human;
use strict;
use warnings;
use base 'DBIx::Class';
__PACKAGE__->load_components(qw/ Core /);
__PACKAGE__->table('human');
__PACKAGE__->add_columns(
    id => {
        data_type         => 'integer',
        is_auto_increment => 1,
        is_numeric        => 1,
    },
);
__PA…

parameterizable packages with Package::Variant

Yesterday I tried to port a Moose App to Moo but got stuck when I found out it is using MooseX::Role::Parameterizable, a module not available in Moo. A quick visit in channel #moose on perl.irc.org later and I was told to give Package::Variant a try.

Few lines of conversation/debugging later and here is my parameterizable HTML::FormHandler role. Package::Variant is an extremly promising module to me that is really helpful when refactoring your code.

declare the HFH role

package MyApp::Role::Form::Step:…

Poor Mans Jobqueue for Catalyst Apps

Handling long-running or heavy tasks inside a a requests is something you should avoid.

  • It blocks the available processes for other requests.
  • browser request timeout
  • ...

Instead of using one of the job queue implementations available for perl

  • Resque
  • ActiveMQ
  • ZeroMQ
  • Gearman
  • TheSchwartz
  • ...

i decided to reuse/abuse my database.

Add a 'job' table


CREATE TABLE "job" (
  "id" serial NOT NULL,
  "test_id" integer NOT NULL,
…