Combining Syslog and DBI

Saw an interesting interaction today. The Sys::Syslog module makes use of fork to prevent a blocking write to the console. Unfortunately, this means that if you are connected to a database via DBI and try to log anything via syslog, you'll run the issue of the DBI handle being closed under you. Even the DBI property of InactiveDestroy cannot save you b/c of the fork being hidden in Sys::Syslog. Thanks to David E Wheeler for the DBI AutoInactiveDestroy attribute which saves the day!

VS2010 Is Too Old?

> cpan Win32::Console::ANSI

ANSI.xs(12) : fatal error C1189: #error : your SDK is too old... (see the README file in the distro)

Fix:
@Set INCLUDE=C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include
@Set INCLUDE=%INCLUDE%;C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE

Perl Epigraphs

Many Perl release announcements included an epigraph, a short excerpt from a literary or other creative work, chosen by the pumpking or release manager. This file assembles the known list of epigraph for posterity, and also links to the release announcements in mailing list archives.

It's quite interesting to read through the epigraphs in epigraphs.pod - a small insight into the Perl pumpkings. How many epigraphs do you recognize?

Installing Local Perl And Libraries On Mac Book Snow Leopard

I recently installed Perl 5.14 on a new Mac Book running Snow Leopard so I thought I'd share my findings and experiences to save the next person doing this some time.

I started off by doing a search on Google for "install perl mac".
Hit number 1: Download Perl - www.perl.org http://www.perl.org/get.html

That suggests you have a perl already installed, which is currently perl 5.10, or you can use App::Perlbrew.

It wasn't quite that easy so I've documented below exactly what I did. I've given feedback to Leo Laporte who looks after http://xrl.us/installperlosx and who should be updating the instructions on the perl.org page. At that point something like "curl -L http://xrl.us/installperlosx | bash" will work.

one-liner for separating iCal events by category

Attempting to move a friend from Outlook to Google Calendar, we exported the calendar to an iCal file.

I wanted to create separate Calendars for each event category.

I searched cpan for modules to read the file and found iCal::Parser, Text::vFile, and Text::vFile::asData.

I really didn't need to parse the data, because then I would have to figure out how to unparse it, and i didn't want the data to get messed up in the process.

I was definitely justified: simply doing iCal::Parser->new->parse($file) put my CPU at 3.5 load for 2+ minutes, whereas this one-liner takes a fraction of a second:

perl  -MIO::File -nE 'if(/BEGIN:VEVENT/){ warn("event not ended: $e") if defined $e; $e = $_ } elsif(/END:VEVENT/){ $e .= $_; $e =~ /^CATEGORIES:\s*(.+)\s*$/m; ($n=$1||"")=~s/\W+/_/g; ($fh{$n} ||= IO::File->new("cal-$n.txt", "w"))->print($e); $e = undef; } else { (defined($e) ? $e : $v) .= $_ } END { print $v }' outlook.ics > leftover.txt

Then a bit of verification:

A Pink Camel in the wild

Poor camel. Who would do such a cruel thing?

pink_camel.jpg

Also, that monkey looks like it's about to take a crap on Amsterdam.

Perl 5.12.4-RC1 is now available

Pheasant is pleasant, of course,
And terrapin, too, is tasty,
Lobster I freely endorse,
In pate or patty or pasty.
But there's nothing the matter with butter,
And nothing the matter with jam,
And the warmest greetings I utter
To the ham and the yam and the clam.
For they're food,
All food,
And I think very fondly of food.
Through I'm broody at times
When bothered by rhymes,
I brood
On food.

-- Ogden Nash, "The Clean Plater"

I've just uploaded the first release candidate for Perl 5.12.4 to
PAUSE. You should be able to find it at:

http://search.cpan.org/dist/perl-5.12.4-RC1/

Here are the changes:

http://search.cpan.org/dist/perl-5.12.4-RC1/pod/perl5124delta.pod

SHA1 sums for this release are:

c46cfc3016367115898d9de8d61847d090b6bd18 perl-5.12.4-RC1.tar.bz2
820fecbd90f645306818a92bffdd5af585ad41df perl-5.12.4-RC1.tar.gz

It's very important that you test this release with any software written
in Perl which you use or maintain. If no "showstopper" class bugs are
found in the next few days, we will release a virtually identical
tarball as Perl 5.12.4.

Cheers, Leon.

App::PipeFilter for Top-N Reports

I'm testing some software at work by replaying pcap files at the application. I want to make sure the results in the database match what's in the original packet dump. There are hundreds of packet producers. I want to focus on the top ten to make better use of my time.

I've written a utility to dump interesting packet data as streams of JSON objects, one per packet. Each object includes the source and destination IP and port, among other things.

% jcut -o src_ip -o dest_ip deleteme.json | head -3

{"dest_ip":"10.10.91.77","src_ip":"10.16.250.39"}
{"dest_ip":"10.10.91.77","src_ip":"10.0.250.80"}
{"dest_ip":"10.10.91.77","src_ip":"10.90.250.39"}

A "top N" report for any single field is trivial. Extract values for that field and pass them to sort(1), uniq(1) and tail(1)

CPAN Testers Summary - May 2011 - Little Lights

May has been a bit of a quiet month, although a few things have been happening in the background. The continuing work of packaging the websites is ongoing, and the plans for the Admin site are coming together. Having said that we did pass the 12 million reports mark in the middle of May too :)

We had a few reports that the testers themselves acknowledged were inaccurate. Although the new Admin site will eventually allow testers and authors to tag reports, the site isn't quite ready for release yet. As such, I now have a script, which uses the guts of the forthcoming Admin site, to hide broken reports. The reports are not deleted, as they may be of use in the future, but hidden from the statistics calculations. If your smoker does send a selection of bad reports, please let me know, and I'll see if I can hide them. As mentioned, the Admin site will eventually take on this role more officially.

Introducing metacpan.org - A better search for the CPAN

I'm very excited to announce the front end for MetaCPAN: metacpan.org . This is what you can expect from the new interface to the CPAN:

Hey Nelson, or, Perceptions

Hey Nelson,

While I’m delivering the news, here’s something for you ignorant American backwoods motherfuckers. Some people’s names have “special characters” in them. Like François Rabelais or Björk Guðmundsdóttir or 艾未未. It’s 2011; the only software that can’t handle Unicode properly is Perl. (As if you needed another reason not to use Perl.)

please quit being an ignorant backwoods motherfucker and stop talking shit about crap you don’t know anything about.


To the Perl folk reading this — the problem we’re dealing with in terms of perception nowadays is confirmation bias. Nelson hates Perl, sees one question on StackOverflow that is making the rounds because Tom answered with one of his obsessively detailed (and therefore huge) missives, generalises wildly from a shallow read of the QA, and then – surprise – finds his opinion further confirmed.

Installing IO::All on Win32 (no Cygwin)

IO:All 0.41 can be installed successfully on Win32 (no Cygwin), if the test fails and spews something like this:

# Failed test at t/file_spec.t line 28.
# got: '33'
# expected: '34'
# Looks like you failed 1 test of 27.
t/file_spec.t ........
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/27 subtests

The alternatives:

  • fix your %PATH% of more than one partition character (';')
  • fix t/file_spec.t

the "everything is a file" metaphor

Frankly I never got it until recently. Sure, I had some informal contacts with plan 9, the OS that is entirely designed around that concept. First when Linus talked about the features he borrowed for Linux, later I borrowed myself features from Plan 9's editor acme for Kephra. But yesterday I saw a talk from the author of MacFuse and I really got a hang of it.

And suddenly I realized the implications for a better Kephra that I currently design. When it read and writes FIles it should have hooks to do whatever (aka *). Not only be able to write files on remote machines via ftp/sftp/*. But also use it to literally edit *. I mean, why do I have to use the clunky Wiki editors, when I can have Kephra, where I'm twice as fast and can store the relevant parts of everything i write in the right places for later reuse? Even that box i currently typing in sucks compared to any decent editor. And please with the right module (Mechanize and friends) its a handful lines to do that.

I really enjoy this (4th or 5th) rewrite.

Display your data - Random::PoissonDisc

I read this nice article on map generation and naturally want to write something like it, but in Perl. For the first step, I want to distribute points nicely across the plane, by using the Poisson Disk Sampling method outlined in the article. Implementing the algorithm was straightforward, but how would I know that the output data was correct in the sense of reaching my goal of uniform but randomly distributed points across the plane?

The test program of Random::PoissonDisc conveniently outputs the generated points to STDOUT. For visually inspecting the data, I can then pipe it into App::ffeedflotr which displays a nice point cloud. The first output of Random::PoissonDisc looked like this:

perl -w bin/random-poissondisc.pl -r 10 | ffeedflotr.pl --type=scatter

Image of wrong distribution

Perl + MongoDB => Mongoers + Fun

For our June 2011 meeting, Samantha Kosko will be talking about MongoDB.

MongoDB is a new document-oriented database. It provides a new paradigm for storing and accessing data, one that works very naturally with Perl.

We'll cover the basics of
- What is it?
- Why do I care?
- How do I get it working?
- What about ORMs? (time permitting)

For those interested in hands-on, please bring your laptop.
Instructions for getting started will be posted and we can do some interactive demos.

MongoDB - http://mongodb.org

10gen - http://10gen.com


RSVP at Meetup - http://www.meetup.com/San-Francisco-Perl-Mongers/events/20864421/

Come and code perl with me in sunny Cornwall

We're looking for a senior perl developer to join our team at headforwards, based in the Pool Innovation Centre in Cornwall - we're an agile team using scrum, shiny mac workstations with big screens in a nice new office and an experienced scrum master working on a project with a MASSIVE Telco.

Nice bunch of people, nice location and decent salary.. full details at http://jobs.perl.org/job/13932

[edit] oh yeah and we're interviewing now, and can move quickly to get the right person in and started soon.. no waiting around for sluggish HR or closing dates :)

Kephra Rewrite called xp

I just started a huge and deep rewrite of what I believe is the most useful editor overall. So why throw it away? (most imortant point is at EOF)

First it will still be maintained, a new version just came out today and i still will fix some remaining issues and even add some features. I will do so, till it can be fully replaced.

On the other side some issues are not that easy to fix, in modules with hairy dependencies, you know what I'm writing about. In fact, I have to write some test app eather way to toy with some solutions. So this rewrite comes with not much overhead over things was planing to do for some time.

GraphViz V 3 is under development!

Hi Folks

Leon Brocard has kindly given me co-maint of GraphViz.

The aim will be to provide essentially the same features, but to support all the latest graphviz attributes.

The distro will contain some graphviz manual pages, with the scripts which downloaded them, and the output of those scripts. They use HTML::Treebuilder's amazing look_down() method to extract lists of valid attributes in various contexts. So, the attributes themselves are stored in data/*.dat within the distro.

That means they can be used to validate user options, because the code loads these lists when new() is called.

Moose is not used. My policy is to use Hash::FieldHash for speed in stand-alone modules, and Moose only for apps.

Here's how you can help:

o There's a module which allows multiple 'files' to be shipped in the __DATA__ section of a script. I've forgotten its name.

IPv6 talk wanted

Dear Perl people,

If you know interesting things about how you can work with IPv6 in Perl nowadays, you are more than welcome to submit a talk to YAPC::Europe.

I believe there're lots of pieces to talk about in that field. The talk can be as detailed as the recent answer about using UTF-8 on stackoverflow, or it can be a five-minute lightning talk. Please share your experience!

A/B Testing and Local Maxima (again)

Given the large amount of commentary on my first A/B testing post, I was surprised that there was none for my "A/B testing and local maxima" post. That's when I discovered that when you start a post and publish it several days later, if you don't update the date, it keeps the start date and thus gets buried in the queue. So, if you haven't subscribed to my atom feed, you may not have seen it on the front page. This is just a heads up that I took the time to answer some questions people had.

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl with a graphic design donated by Six Apart, Ltd.