Phillip Smith

Over the last few years, I've come to rely on tools that summarize information for me: Being on a mailing list in digest mode and receiving a summary of activity once or twice a day is a great example -- and it's also the specific challenge that I'm struggling with.

As my information consumption habits change, I find that I also want to customize how that information is delivered, and when, more than simply changing to digest mode allows. And, frankly, even in digest mode, I receive too many individual e-mails.

To resolve this, I end up setting my list subscriptions to "no mail" and fooling myself into thinking I'll peruse them on the Web from time-to-time. However, for those mailing lists, I'd actually would like to see what's happening regularly. So, last week I started brainstorming a tool that would send me a daily summary of all activity across a variety of different lists -- combining, say, all lists about security into one daily e-mail summary with links to the individual posts.

So, here's the challenge: half the lists are Mailman lists, the other half are Google Groups; half of them are public, the other half private.

After I'd sketched out the initial requirements, I thought to myself "I probably don't need to start from scratch, I'm pretty sure that my old friend Plagger -- the Perl-based "UNIX pipe programming for Web 2.0" -- can do most of this." Thus, I updated my Plagger source from Github and started digging...

Turns out that Plagger can handle two requirements already: consuming a Mailman archive and outputting it in a variety of ways, and -- obviously -- consuming the XML feed that's provided for a public Google Group. In the Plagger world, that would be as simple as this:

So, that part is surprisingly simple. The bigger challenge is dealing with private mailing lists, where:

• If it's a Google Group, I need to authenticate to Google first (though, I only need to authenticate once for all lists, thankfully);
• If it's Mailmain, I need to authenticate to each list with a different password.

So -- drumroll please -- here's the actual question: Should I simple develop a couple of new plugins for Plagger to handle Google Groups and private Mailman lists? Or should I develop something smaller that is focused on my specific set of requirements?

I think the question is really about the status of the Plagger project and community. I've developed a couple of plugins for Plagger already -- one that integrates bit.ly and another that fixes problem in the existing Publish::MT plugin -- but there are a couple things that bother me about going down this road again. Specifically:

• It's difficult to get help for Plagger: the IRC channel is not really active, and the Plagger Google Group is full of spam.
• The Plagger documentation is woefully incomplete and -- in some cases -- just plain wrong.
• The existing Plagger Web site is terribly unhelpful, and more-often-than-not, contradictory (where do I get the latest Plagger source? From Subversion, or from Github?)

Lately, some folks have told me they looked at Plagger and then decided to develop their own aggregation tool instead of using it. So, What am I missing? Should I be heading down the "roll my own" path also?

There seems to be some recent activity on Plagger's github page -- a few forks anyway -- so the project is not entirely abandoned, it appears. However, it would be really comforting to see some efforts at making it easy for those of us who are working with Plagger to share configuration files, new plugins, and so on. Or maybe none of this should be a concern at all?

What do you think? What would you do?