YAPC::Europe Day 3

The third and last day of YAPC::Europe and everyone is a little tired after all the socialising and thinking.

This morning there was a little presentation about YAPC::Europe next year in Kiev, Ukraine. Should be cooler!

Stevan Little presented A MOP for Perl 5 which sounds like he's chosen a very simple syntax for nice concise OO code (and a backend).

Kenichi Ishigaki presented CPANTS: Kwalitative website and its tools where he explained the story behind taking over CPANTS and improvements he has made.

Shawn Moore presented Moose Role Usage Patterns to a very packed room where he explained some useful patterns behind roles.

Philippe Bruhat presented Organizations of the Perl community where he listed the many and varied Perl organisations, how they came to be and what they are doing now.

Mark Keating presented Adventures in Marketing: Part Two where he explained marketing Perl over the last year and plans for this year, such as learning Perl, a Perl desktop environment and more.

The goodness of testing

This is not strictly related to Perl, but I was thinking about my practice of testing recently and I've come to some conclusions, some of them proving to be a bit controversial. Read more here and have your say!

Precedence parsing made simpler

This post describes a new approach to precedence parsing, one that makes it simpler and more flexible. Many programmers find precedence is an intuitive way to look at problems. The traditional rules of arithmetic are a familiar example:

E ::= ( E )
E ::= n
E ::= E * E
E ::= E + E

Here, as in the rest of this post, the rules are ranked from tightest ("highest") precedence to loosest ("lowest"). The order in the above display indicates that multiplication takes precedence over addition, and parentheses take precedence over everything else.

The old way and the new way

The traditional way to deal with precedence centers on symbols. The symbols are divided sharply in two: those that define structure and those that carry information. The structural symbols (often called "operators") are assigned associativities and precedence. To help guide the parse, symbols may be further classified as infix, prefix, circumfix, etc.

Update to review of Dependency modules

I've updated my review of modules for getting dependency information. The previous version listed 8 modules on CPAN — it now covers 18 modules!

The following modules have been added:

  • App::FatPacker::Trace
  • Devel::Modlist
  • Devel::TraceUse
  • Dist::Requires
  • Module::Extract::Use
  • Module::Info
  • Module::Overview
  • Module::PrintUsed
  • Module::ScanDeps
  • Module::Used

One of the recommendations has changed: if you want an approach based on parsing, I'd now go with Module::Extract::Use rather than Perl::PrereqScanner. There's not much between them (both use PPI), but Module::Extract::Use returns a bit more information.

Given I'd missed 9 modules in my first scan of CPAN, and then missed 2 on my thorough search, I wouldn't be surprised if there are still a few to add...

Update 2012-08-27

Indeed: I've now added Module::Inspector, Module::ParseDeps and Module::MakefilePL::Parse, bringing the total to 21 modules. I also updated the section on Devel::TraceUse, following email with Philippe Bruhat.

YAPC::Europe Day 2

Yesterday's attendees dinner in Sachsenhausen was excellent and we did indeed have Apfelwein.

The day started off with Curtis Poe explaining to us the agile manifesto. I think he was trying to get us to all launch startups or at least reengineer our current employments.

Then Clinton Gormley presented To infinity and beyond: Storing your Moose herd in ElasticSearch" where he talked a little about ElasticSearch, a little about scaling and a little more about object mapping.

Sawyer X presented Asynchronous programming FTW! where we went through IO::Async, POE, and AnyEvent and how to think in callbacks.

Bernd Ulmann presented Array programming for mere mortals where he scared us with APL.

Alberto Simões presented Building C/C++ libraries and applications with Module::Build‎ and went through all the options.

Then it was time for my talk: The Fallacies of Distributed Computing which was a general talk about how network are useful but there are some things you need to keep in mind.

I'm currently in Matt S Trout's Distributed Daemon Discovery which is a nice little story.

After that we have another hour of lightning talks and then off to Beers for Engineers...

Building C and C++ libraries with Perl

It seems that somebody screwed it, and forgot one 20 pages article I wrote for YAPC::EU::2012 proceedings. Therefore, the proceedings are shorter, and are shamelessly incomplete. Anyway, you can read my article in all its glory here . And sooner or later I'll add the presentation too.

A NYTprof encoding hiccup

While using Devel::NYTProf on a new application I started getting this message

fid 33 has no src saved for /usr/lib/perl5/5.14.2/autodie.pm (NYTP_FIDf_HAS_SRC not set but src available!)

Now my first thought was this has something to do with either the newest version of autodie or utf8::all. So I checked to make sure all the modules I was using were up to date and tested again, still there. Then I wrote a really short program to recreate this error and for some reason I couldn't. Going back and forth between the two files I finally noticed what was different and I was shocked by what it was.

This script generates the above message with `perl -d:NYTProf`

and this one does not.

Just switching the order of autodie and utf8::all fixed this. My mind was blown. I could not think of another time I had seen the use order do something like this. So I did some searching to see if this was known or not. Bug 70211 says it is 'use open qw( :encoding(UTF-8) :std );' followed by a use module statement that causes the issue. So my "fix" is not really a fix but there appears to be no actual problems with nytprof but I am keeping an eye on it.

Devel::Trepan 0.31 release

A new release of Devel::Trepan is out there.

A number of changes were the result of problems found in writing the various debugger blogs and in preparing for a little local NY Perl Mongers meeting.

Possibly there will be one more debugger blog on stepping.

YAPC::Europe Day 1

It's the first day of YAPC::Europe in Frankfurt and it's the hottest day of the year so far. It's pretty warm in the conference venue. Max gave us a good introduction to the conference and handed over to the announcement of YAPC::Europe next year: Kiev, Ukraine. Then the star of the show: Larry Wall took a small Perl 5 script and slowly converted it to Perl 6 live, showing some useful debugging features and nice compact clear code.

David Leadbeater presented An exploration of trie regexp matching where he explained the inner workings of Perl's regular expression engine and RE2.

Damien Krotkine presented Dancing with
WebSockets
digging into the live web.

I really enjoyed Lenz Gschwendtner's talk on Continuous deployment with Perl on how to build robust deployment and deploy all the time.

Then I went to sleep outside in the shade as it was all getting a little too hot for me. The organisers kindly provided some icecream!

I came back to see Eric Johnson present Selenium testing with Perl which mirrors work I've been doing recently with Cucumber / Capybara / PhantomJS.

After that it's time for the lightning talks, hosted by me. Lots of little talks with lots of speakers: what could go wrong?

Tonight we have the attendees dinner on the other side of Frankfurt. Time for a little Apfelwein...

#perl6 summary for week ending 2012-08-18

Summary entries suggested/written by #perl6 users. Editing by raiph.

(The rest of this blog entry, following this paragraph, was originally a reddit post. I decided to copy it here as a first entry for this new "Perl 6 reports" blog, which is the home for these summaries and other Perl 6 reports from now on.)

I fondly recall the excellent summaries of Perl 6 mailing list discussions Piers Cawley posted back in the day. He kept it going for years. Nowadays the main Perl 6 action is the freenode IRC channel #perl6. This is the first of what I hope will be a regular series of #perl6 highlight posts.

2012-08-11:

2012-08-12:

DIY personal analytics

How many times a day do you reach for <ctrl> + r when using the shell? What about the history command? !! anyone?

Do we as programmers evolve and stop making the same mistakes? Do we really optimize our workflows? This is where the idea of personal analytics comes in. I am going to see what I can learn from looking at my bash history for the last few years. Here are the relevant settings in my .bashrc file:

shopt is a bash command that shows and changes shell option names. The histappend option tells bash to append the history collected to the filename specified in HISTFILE instead of overwriting the file. cmdhist tells bash to save all lines of a multiple-line command in the same history entry.

HISTFILE allows me to tell bash where and what to name history files. For example 2012-08-20.hist is today's bash history file.

#perl6 summary for week ending 2012-08-18

Summary entries suggested/written by #perl6 users. Editing by raiph.

(The rest of this blog entry, following this paragraph, was originally a reddit post. I decided to copy it here as a first entry for this new "Perl 6 reports" blog, which is the home for these summaries and other Perl 6 reports from now on.)

I fondly recall the excellent summaries of Perl 6 mailing list discussions Piers Cawley posted back in the day. He kept it going for years. Nowadays the main Perl 6 action is the freenode IRC channel #perl6. This is the first of what I hope will be a regular series of #perl6 highlight posts.

2012-08-11:

2012-08-12:

YAPC::Europe day -1

YAPC::Europe 2012 in Frankfurt am Main, Germany starts on Monday. This is the biggest gathering of Perl people in Europe and I'll keep you updated day by day.

I'm presenting The Fallacies of Distributed Computing‎ on Tuesday afternoon.

The star of the conference for me is the lightning talk sessions: lots of speakers giving 5-minute talks. R Geoffrey Avery normally hosts them but unfortunately he can't make it this year, so I'll be your host for the lightning talks this year. There are lightning talks at the end of each day. I can squeeze in a few more lightning talks: if you'd like to give one, come see me and send me an email: acme@astray.com.

A comparison of stopword list modules

Hi Folks

I've just released Benchmark::Featureset::StopwordLists.

It compares 3 modules implementing stopword lists.

You can skip the review and just examine the report.

Various other modules have stopword lists, sometimes using one of these modules.

If you know if any stand-alone modules implementing a stopword list, please comment here.

An overview of spell checking modules

Spell checking is one of those problems that is already solved... sorta.

Like all problems it really depends on context. Take Jon Bentley's Programming pearls: a spelling checker where he examines the problem space and the differences between a spell checker and a spelling corrector. I start by searching the keyword 'spell' across all of CPAN.

wget http://www.cpan.org/modules/01modules.index.html
ack -i spell 01modules.index.html

The above covered all 22,442 distribution names but not the sub modules names. A few metacpan searches later and I was able to compile the following list.

Direct checkers - modules that actually do the spell checking
  • Lingua::Ispell A module encapsulating access to the Ispell program via IPC::Open2
  • Meta::Tool::Aspell run aspell for you. Meta is a class library of about 250 classes and is abandonware.
  • Text::Aspell Perl interface to the GNU Aspell library
  • Text::Hunspell Perl interface to the GNU Hunspell library
  • Text::Ispell A wrapper module for Ispell. The ispell cli is called via IPC::Open2.

Ricardo is pushing for smart match changes.

A long time ago, Ricardo suggested big changes to smart matching. I added my thoughts on how that does not benefit Perl. Now he's pushing for those changes again.

This sort of change should be important to all Perl users. We already have two versions of smart match out there so you have to be careful with your Perl versions. This would add a third. I don't think these changes are in the interests of most users, and unless ordinary users pay attention, they'll get whatever p5p decides to give them.

Jesse Vincent proposed moving all smart match changes out to pragmas so you could know which one you would get. He also suggested that new features come in as pragmas before they make it into core.

Lausanne seminar update

Last week I foreshadowed that we would be offering a free evening seminar when I'm in Lausanne next month.

The arrangements for that talk are now finalized. My thanks to GULL for providing the venue, and especially to my good friend Frédéric Schütz for arranging everything.

You can get the full details of the event in the official announcement, but briefly:

What: "Taming Perl Regexes"
Where: Beausobre, Morges
When: Monday, September 24, 19:30.

It should be a fun talk, and (for a change) a very practical and useful one!
I hope to see you there.

Damian

Tech Tip: How to Package and Maintain CPAN Distributions in Mageia

Mageia Linux is an RPM-based Linux distribution, whose repositories contain over 3,000 CPAN packages, and part of the reason why it has so many is because Jerome Quelin and the other maintainers have worked on tools to facilitate creating Mageia packages for CPAN distributions and maintaining them.

However, I was a little confused about using magpie, so I'd like to share my findings here:

  1. In order to import, upload and submit a new CPAN package into Mageia, along with all of its dependencies, one should not use magpie, but rather cpan2pkg. Its use is very simple: make rpm and urpmi sudoable, and type cpan2pkg Package::Name from the command-line inside an X terminal. This will start a Tk window where one can monitor the progress of preparing new RPM packages and it has an entry box to create more packages (which saves time on re-initialising CPAN.pm or CPANPLUS.pm).

  2. In order to upgrade a package, one can type eval $( magpie co -s perl-[PACKAGE_NAME] ) and then magpie update. magpie requires minicpan to be installed and updated.

  3. In order to install packages, one can do sudo urpmi 'perl(Package::Name)'. My Module-Format module facilitates the translation from other notations for writing modules:

    up()
    {
        sudo urpmi --auto $(perlmf as_rpm_colon "$@")
    }
    

Backing up Berlios.de

Last year it was announced that www.berlios.de was going to be shut down. People were asking if someone was going to back it up to save all those open source projects. I decided to gave it a shot and I was able to backup all of the berlios projects. While working on the process of uploading it to a new host (I was looking at github) it was announced that the site was saved, so I set the project aside.

Digging around I found this code and decided to post it so that people who are trying to build data mining style tools can have another real world example. github.com/kimmel/backup-berlios.de contains two scripts, a shared library and a data file.

01_fetch_project_list.pl builds a list of all the projects on Berlios and writes it to a file.
02_download_repos.pl takes that data file and downloads everything it can.

Use exceptions instead of calling croak()

This little bit of test code was causing me a lot of grief:

You see the regex for qr/Table.1111111111.doesn't exist/? Due to a slight rewording in the error message, that test kept failing. However, it was failing in a way that the following test used to keep failing. As it turns out, I had fixed a bug these tests were designed to catch but it looked at first like I hadn't fixed the bug. Because of the changed error message (and me misreading the test number), I spent a lot of time trying to track down a bug that did not exist.

If I had been throwing proper exceptions, my tests would be trying to validate the class of the exception, rather than the text of the exception. I could have changed my error messages at will without worrying about breaking my tests. Yet another reason why you usually want exceptions instead of calling die or croak.

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl with a graphic design donated by Six Apart, Ltd.