Guillermo Roditi

Yesterday and today I released to CPAN Cantella::Worker and Cantella::Worker::Role::Beanstalk. I am very happy with these two packages. They are the culmination of about a month of on-and-off playing around with different approaches to having distributed event-driven worker processes. As an added benefit, I can also run a pool of beanstalkd servers in case I need to bring one down for maintenance or I have some unexpected downtime.

I am aware of the existence of MooseX-Workers, but it didn’t ‘feel’ right for the job. Additionally, the MooseX-Workers approach of running jobs in a separate process via POE::Wheel::Run didn’t work very well for my approach. You see, in order to reserve a job from beanstalk, a connection must be open with the server. During a fork, I could either fork and keep file handles open, making sure the file-handle survived, or lose my connection and my reserved job. Additionally, the constant forking was a little too much over-head for the very simple jobs I do.

Instead of trying to make MooseX-Workers do what it wasn’t designed to, I decided to go with a preforking approach and roll my own approach. I have a manager process and n children processes. Children processes are free to work by themselves and block if they want to. Communications between the manager and worker processes happens via POSIX signals. Right now 3 commands are supported, pause (USR1), resume (USR2), shutdown (TERM & INT).

So far, the preforking manager is pretty basic, but I look forward to adding the following in the future:

• Detaching children that terminate themselves if the parent dies
• User, group, priority adjustments in the child process
• More real world examples.