Inline Granted

This news is a couple of weeks old now, but the Inline-for-XS-Modules TPF Grant Proposal was accepted! A big Thank You to the Perl Community and the TPF.

David and I have just returned from vacations and we started work on the project today. Our first order of business was to create this web site, to which we will be posting regular updates on our progress, and other interesting information about the project.

A fast pragmatic test runner

After breaking the build twice in the space of a week for more or less the same reason each time, I figured that my team needed a continuous integration rig, and we needed it now.

For web application stuff it's not that unusual to have tests that collide with each other, at least in the short term due to the conflict between the need to "do it right' and the need to "do it now". And generally it's desirable to run your test suite in parallel wherever possible. At the moment, our tests are generally inadequate, so I'm only saving around 2 minutes on a parallel versus series run. However I'm doing bulk changes (with the help of PPI) to this largeish existing codebase at the moment, and get into situations where I want to run the whole test suite every 10 minutes or so, at which point saving 12 minutes in an hour becomes significant.

Reviewing gitolite modules from the puppet forge

When a million flowers bloom in the world of open source, with no clear winner in the popularity contest for wide-spread use, sometimes the options can overwhelm. What follows is a summary of what I learned spending a day reviewing my options for a tool I could perhaps have written myself in that time, if I had felt free to ignore the prior art. It relates to yet another great tool written in perl, gi‪tolite by sitaramc.

-----

If you manage your infrastructure with puppet, your source code with git and have need of an in-house git repository providing user and group access control lists, this article may be relevant to your needs.

So I went to Mongolia and rode a Camel

For some reason I just felt this deserved to be posted here.
I went to Mongolia and rode this bactrian camel. As a JAPH this was a special event for me worth sharing :)

1978841_10152342517411771_6701857937254181945_n.jpg

Language Identification, Neural Networks and Perl

I will not write much, but just would like to let you know there is a new Perl module for Language Identification (Lingua::Identifier). It uses a neural network for the task (read this for details), with Math::Matrix::MaybeGSL, that will use Math::MatrixReal or Math::GSL::Matrix if it is installed. An extended version of the paper is being reviewed, and therefore I am not allowed to publish it here.

Q: Nestoria Dev Blog and comments

On Nestoria Dev Blog I tried to post a comment+script about smart match.

  • I log on via Google
  • Everything looks ok.
  • I post text + script
  • The page refreshes
  • Everything looks ok

But the post is not afterwards visible.

OK, so it’s possible the blogger just zaps my post :-(, but it there another explanation?

Fatpacking your scripts with fatten

"Thank you for a freakin awesome app. Seriously, fatten is great! <3" --Paul J. Fenwick

fatpack is a utility created by MST to pack a script along with its dependencies (required Perl modules) into a single file. It differs from PAR in that it does not create an archive that must be extracted into the filesystem at the start of program run. Thus, a fatpacked script is simpler and faster to run.

The command-line utility is a bit cumbersome to use, though. Producing the final result involves several invocations of fatpack, and often the final result still misses some required modules. If you want a more turnkey solution to fatpacking a script, you might want to check out fatten.

Using fatten could not be easier:

% fatten /path/to/myapp

Parsing of anime reviews on rec.arts.anime.misc

I have been collecting posts from rec.arts.anime.misc newsgroup, mostly related to anime reviews to prioritize which one to see. The format is more or less the same of (collection of) reviews ...

  • title
  • a paragraph of comments
  • ending with some kind of quality statetment (a second paragraph is rare)

Take ca6u74F3g1tU1@mid.individual.net message for example by GeoffC (Wed, 15 Oct 2014 10:48:16 +0100) ...

Gundam G no Reconguista:
Another Gundam, still with young characters piloting giant robots in space. Not badly done, but I was confused about who was who and what was going on in the first episode, and did not feel sufficiently involved to continue with further episodes.

...

Donten ni Warau:
Meiji Restoration period drama. I saw the first episode in raw. So far, it has not interested me enough to encourage further investigation.

Visualizing the McNaugthon-Yamada-Glushkov Construction

Once in a while I poke in my old Perl stuff to find something interesting. Luckily, this is usually the case.

Today, I found a script that draws an annotated syntax tree of a regular expression. The annotation shows the state sets calculated when applying the McNaugthon-Yamada-Glushkov algorithm to construct a finite automaton from the regular expression.

You can read about it in "Bruce W. Watson. Taxonomies and Toolkits of Regular Language
Algorithms. PhD thesis, Faculty of Computing Science Eindhoven
University of Technology, The Netherlands, 1995".

Here is an image:

myg.png

I just thought about sharing this bit because there once was a thread about how we could improve marketing for Perl and one of the answers was to write (more) Perl applications.
So, here is one :)

Sydney Perl Mongers Thursday Night

Sydneysiders are invited to join us this Thursday night for yet another Sydney-PM meet.

SiteSuite have offered to host us. Thanks to Cees for setting the wheels in motion for us to use the venue.

What: Sydney PM
Date: Thursday, 16th October 2014
Time: 6-9:30pm
Where: SiteSuite, Level 3, 1 Bay Street, Broadway Shopping Centre, Sydney
Who: Anyone with any level of experience with Perl. Please bring friends, family and co-workers!

The building locks it's doors at 6pm, so one of their staff will let us in. On the day a contact mobile number will be posted for stragglers. Please jump on the sydney.pm.org email list for that detail on the day.

Cees will be giving a talk about Rose::DB and we have a guest speaker giving us this talk live from San Francisco...

Title: I've Got 99 Problems But CPAN Ain't One

Celebrating Perl 5’s 20th birthday at MadMongers.



Celebrating Perl 5’s 20th birthday at MadMongers.

[From my blog.]

Pod::Readme v1.0.2 released

I released a new version of Pod::Readme yesterday. It's a module for "Intelligently generate a README file from POD" by using POD =begin/=end and =for commands to control what parts of a module's POD are included/excluded in the README. For example, you don't need the details of method calls in the README, nor do you need installation instructions in the module's man page.

This is a major rewrite using "modern Perl" that supports the following features:

  • Generating a README in various formats such as plaintext, POD, markdown, HTML;
  • Support for plugins to add new features.
  • Support for (via plugins) inserting the module version, recent changes and prerequisites in the README.

It should still work with existing software that uses it, such as Module::Build.

The next steps will be to write modules to integrate this with other release tools, such as Module::Install or Dist::Zilla.

Feedback as always would be appreciated. The module is on GitHub.

Thanks to everyone who commented on previous blog posts about this, and to CPAN Testers for test reports on previous dev releases.

A most amusing annoyance

Came across this in my travels this morning:

$ perl -E 'sub x { say "y"}; my $x = 'x'; $x->();'
y
Undefined subroutine &main::1 called at -e line 1.

Why on earth does this compile and run in the first place? Fixing the shell quotes makes the problem go away.

Which modules are currently loaded by process?

Have you ever wondered which modules are currently loaded by current process? With Perl it's very simple to know this piece of info. Perl saves loaded modules in a hash %INC.

Lets have some dirty hands:

use Data::Dumper;
print STDERR Dumper \%INC;

The output is as follows:

$VAR1 = {
          'warnings/register.pm' => '/usr/share/perl5/warnings/register.pm',
          'bytes.pm' => '/usr/share/perl5/bytes.pm',
          'XSLoader.pm' => '/usr/lib64/perl5/XSLoader.pm',
          'Carp.pm' => '/usr/share/perl5/Carp.pm',
          'Exporter.pm' => '/usr/share/perl5/Exporter.pm',
          'warnings.pm' => '/usr/share/perl5/warnings.pm',
          'overload.pm' => '/usr/share/perl5/overload.pm',
          'Data/Dumper.pm' => '/usr/lib64/perl5/Data/Dumper.pm'
        };

The hash simply contains the module path as value.

So this can be used for cleaning up too, with it you can figure out why you're not getting the version of the module you're expecting, for example.

Madison Area Perl Mongers Tonight

Tonight’s topic: REST on your SoC with Device::WebIO

System on a Chip (SoC) devices, such as the Raspberry Pi and pcDuino, are becoming increasingly popular. They almost always have a working Perl installed, and some library for controlling the I/O pins. Wouldn’t it be nice to unify the API across devices? While we’re at it, why not add a REST interface, too? The Device::WebIO modules do exactly that, and this presentation will show you how to use them.

This is also the 20th anniversary of Perl. There is a rumor of cake.

[From my blog.]

Synchronizing Opera bookmarks with Perl, Org, and git

Who here like me still uses Opera (specifically Opera 12 on Linux)? It's being abandoned, the bugs are piling up, and more websites are not rendering correctly on it. However, the combination of keyboard shortcuts and some specific features like editing+applying source code makes me still stick to it.

The Opera makers provide a service called Opera Link that can synchronizes your browser bookmarks and a few other stuffs between computers and mobile devices. However I prefer not to use it and rely on some good ol' tools instead.

I also happen to be a fan of Org so Org will be the master document.

The steps that I describe here might be too much of a hassle, but I like it and thus share it.

First, you install the CPAN module App::ConvertOperaBookmarksToOrg which includes the utilities adr2org and org2adr.

Then you convert your Opera bookmarks to an Org document with this command:

% adr2org -T ~/.opera/bookmarks.adr > repo/opera-bookmarks.org

A Grand, 2014

A couple of days ago I got a tip that I was very close to being the first CPAN author to upload 1000 releases in a year (989). Today I topped it off:

Thus ends my CPAN numbers ambitions. From now on I will concentrate on -Oquality -Ofun and -Oacmeism. Actually the numbers stuff ended in early September.

The -Oquantity and -Ofrequency flags were fun for a while, and it resulted in my getting a large portion of my modules into an up to date state; but numbers for numbers sake has a limited appeal.

Note: It was ironic that one of my final releases was from a pull request by SHARYANTO++ who leads almost all the CPAN statistics. :-)

Getting IP address with Perl

A common situation which you will usually come across, is to retrieve the IP.

Using WWW::curlmyip, this task is very easy.

use WWW::curlmyip;
my $ip = get_ip();

Lets inspect the module on https://metacpan.org/pod/WWW::curlmyip closely to understand how it works.

use strict;
package WWW::curlmyip;
$WWW::curlmyip::VERSION = '0.02';
use 5.008;
 
# ABSTRACT: Returns your ip address using L<http://curlmyip.com>
 
 
BEGIN {
    require Exporter;
    use base 'Exporter';
    our @EXPORT = 'get_ip';
    our @EXPORT_OK = ();
}
 
 
sub get_ip {
    my $response = HTTP::Tiny->new->get('http://curlmyip.com');
    die join(' ', 'Error fetching ip: ',
                  ($response->{status} or ''),
                  ($response->{reason} or '')) unless $response->{success};
    my $ip = $response->{content};
    chomp $ip;
    $ip;
}
 
1;
 
__END__
 


It's a one file module, which simply contacts http://curlmyip.com, retrieves the content with HTTP GET request, and finally chomps the result; because there exists an extra new line character which is useless.

So simple, isn't it ?
But It's also very useful !


Elasticsearch Custom Scoring

Elasticsearch has a builtin scoring algorithm which works quite well in practice, but sometimes you want to roll your own scoring algorithm. Let's examine how to create a custom scoring algorithm using the function score query.

Let's assume we want to search and score Perl job offers based on a set of weighted keywords. To get the score of each offer, we'll multiply the weights of matching keywords where positive keywords have weights greater than one, and negative keywords are weighted less than one. Thus, the positive and negative keyword matches add and take away from the final product (score) respectively.

Define the Qualitative Importance and Their Weights

Qualitative Tags

Instead of assigning weights directly to keywords, let's use qualitative tags of desirability (importance) for each keyword (trait). Borrowing from okcupid's five degree scale of importance:

  • mandatory
  • very
  • somewhat
  • little
  • irrelevant

This takes care of the non-negative traits, now let's add qualifiers for negative traits using a similar pattern:

  • mandatory_neg
  • very_neg
  • somewhat_neg
  • little_neg

Actual Weights

Final TPF Devel::Cover grant report

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl with a graphic design donated by Six Apart, Ltd.