Kal

• About: I stuff with perl and stuff.

Recent Actions

• Commented on Thread pool for a boss/worker model
  beanstalk seems to be very much like Gearman. Its a TCP client/server model in which you feed jobs into 'tubes' or queues on the server, and workers which join to the server and then consume those jobs. You can't register...
• Commented on Thread pool for a boss/worker model
  Wow, I didn't see these, I'm still used to looking for relatively self-descriptive names I guess. :) Gearman is definitely lean in terms of dependencies, but its single threaded and I'd still have to manage multiple worker threads (and a...
• Commented on Thread pool for a boss/worker model
  @happymorning - I thought about using an MQ for the queues/thread control/boss messaging since the org is using a lot of ActiveMQ already, but this is pretty simple - a process from the UI or from someplace else says "Take...
• Posted Thread pool for a boss/worker model to Kal

  This is a pretty simple idea - a boss thread assigns work to a pool of worker threads who do nothing until some work enters their queue. This way the boss can fill a queue very quickly and you have multiple back end processes that can consume that queue.

  I'm using threading and not an asyn…

• Commented on Acceptable Community Standards
  Its really nice to see this sort of feedback. I'm glad that others feel the same way. These sorts of ugly incidents may have the reverse effect as I'm seeing the really good side of the community and may try...
• Posted Acceptable Community Standards to Kal

  I've been lurking here for months - there are some serious experts that blog here, along with a few absolute newbies. I figured my experience level didn't matter. I could have blogged elsewhere too....But...

  I'll admit that part of the reason I chose to write my blog here is due to the rec…

• Commented on my $perl_blog;
  Thanks for the welcomes! @vti/Ranguard - Yep, gone through all those sites, and Task::Kensho is a fantastic reference for CPAN modules! @Joel - I just started using the MongoDBI code in a project, its an interesting idea and requires some...
• Posted my $perl_blog; to Kal

  I use perl, at home, at work, and I've been using it for over a decade. I recently switched job titles and had to relearn The Web outside of the old CGI forms that I used to know back in the crusty old days of web 1.0.

  Handrolled mysql connections and 50k lines of code aren't good practic…

Comment Threads

• Ron Savage commented on Thread pool for a boss/worker model

  Hi Kal

  What about these Perl modules:
  1) Gearman
  2) Hopkins
  3) TheSchwartz
  4) Helios (which uses TheSchwatrz)

  In fact, it'd be marvellous if you'd try all of them and write a comparative review for those of us who haven't actually tried any...
  

• rhesa commented on Thread pool for a boss/worker model

  Another suggestion: beanstalkd and the Perl module Beanstalk::Client.

  I use this by forking off a number of workers per machine. Then I add more machines as load increases. Having isolated processes that only talk to the beanstalkd server means I don't have to deal with synchronisation or blocking issues. This makes for very simple and efficient code.

• chimerix commented on Thread pool for a boss/worker model

  Have you seen Parallel::Workers or Thread::Pool on CPAN?

• renodino commented on Thread pool for a boss/worker model

  Gratuitous self promotion: Thread::Apartment provides a pooling interface, and should allow use of direct method calls to threaded objects.

• Ron Savage commented on Thread pool for a boss/worker model

  Hi Kal

  Beanstalk::Client is written by Graham Barr, which is a big plus in my books.

  BTW: I haven't used any of these, but would be v-e-r-y confused if confronted with the need to to do.

  As for overlooking those modules I listed, I'm not surprised. Many modules have obscurantistic names.

  That's why I keep a tiddlywiki (check them out!) (one of many) for Perl with a section dedicated to 'Interesting Modules' I stumble across.

  I guess it's time to calve out that list into its own web page and put it on my site. I'll do that over the next week and blog on…