blogs.perl.org

Hi Folks

I have just finished a re-write of CGI::Session, called Data::Session, and started wondering how best to test it.

I've always disliked setting all the env vars to make its tests work for MySQL, PG, etc, so I hit CPAN looking for A Better Way.

I found Test::Database, which seemed to be a good idea. But examining the code was discouraging, so I decided to re-write that too.

I soon realized that one thing (among many - see below) which bugged me about Test::Database was its handing of DSNs.

So I decided to split that out into a separate distro, using an INI-style file, so I could add extra attributes to the DSN, such as active and use_for_testing. I might add use_for_production too.

That's why I've just released to CPAN DBIx::Admin::DSNManager V 1.00.

Now I'll use that in the re-write of Test::Database, which I'm going to call Test::Admin::Database.

Padre 0.74 has been uploaded to pause and is in the queue for indexing and replication to your favourite CPAN mirror.

So while all that happens, lets take a look at what has been happening since 0.72.

Well, plenty of fixes, and a fair amount of work done with the Plugins and converting them to Dist::Zilla by Ahmad Zawawi. The changes file doesn't really show the work that goes into the Plugins, but keeping one eye on the irc logs shows plenty of commits from Ahmad between releases.

Many of the plugins for Padre reside in the same SVN repository as Padre itself, so if you find an itch that needs scratching, you don't have far to look to start scratching.

Zeno Gantner is finding time to work on bits of Padre even with a rather hectic sounding life at the moment.

I'm about to start writing about a bunch of stuff that will definitely show my lack of a computer science background. Unlike many of my posts, this is your chance to correct me rather than me explain things to you. This has been on my desktop for awhile, so I'm cutting bait and posting what I have instead of working on it more.

I've been reading Seven Languages in Seven Weeks: A Pragmatic Guide to Learning Programming Languages, which is an enjoyable book except for the parts where he starts to talk about types and, ahem, types of programming languages. It's mostly distracting, not very useful, probably misguided, it not outright wrong.

He ist lead dev of Scintilla the edit component, Kephra, Padre and Kommodo are using. And I was very happy to read that he will open up the internal API. So Perl editors will be able to do even more.

Exciting.

After a bug report from a user, I've uploaded a new Data::Google::Visualization::DataTable.

You can do some fun stuff with this module - it allows easy preparation of data for use with the Google Visualization API - ie: any data source you can access in Perl, you can easily spit out as one of a number of awesome and often interactive graphs and charts.

If you make something pretty with it, let me know!

Yesterday we had some troubles setting up a client that should post some utf8 JSON to a web API. The problem was, that the data kept showing up as latin-1. After lots of "fun" with hexdump, wireshark, the debugger and Devel::Peek we verified that we were in fact passing a properly utf8 encoded JSON to LWP::UserAgent like so:

    my $response = $ua->post(
        $self->api_base_url . '/' . $args->{action},
        'Content-type'   => 'application/json;charset=utf-8',
        Content          => $args->{data},
    );

Still we didn't receive utf8 on the server.

After adding encode_utf8 in random places, it worked. So I dug into google and found this posting containing this advice:

If you want the string UTF8 encoded, then say so explicitly:

 $req = HTTP::Request->new(POST => $endpoint);
 $req->content_type("text/plain; charset='utf8'");
 $req->content(Encode::encode_utf8($utf8));

It seems that HTTP::Request, when stitching together the actual text representation of the request, will downgrade the content to latin1, no matter what you say in Content-Type, unless one explicitly calls encode_utf8 on the content.

Top 10 Mistakes When Building and Maintaining a Database is a good overview of what not to do when creating a database.

The only caveats I offer about this article:

• The article is SQL Server-centric; and
• Mistake #4, "Use of Adhoc Queries", seems to be true only when you have a large, heterogenous development staff (like 10+ developers). Smaller development groups don't seem to suffer from allowing ad hoc queries by developers. (Perhaps it is mainly a communications problem, rather than just an allowing ad hoc queries problem?)

I've always been an avid hater of uninitialized variables. I'll hunt them down, catch them and beat them to a bloody pulp and then shoot the remains.

Too much, you say? No need for the bat AND the gun, you say? Oh, quite the contrary.

Here is an example of a recently discovered Linux kernel issue. An uninitialized memory segment. The fix is really simple: to initialize it with zeros. This was done in a patch that only adds the following command:


+ memset(mem, 0, sizeof(mem));


Oh, but I can hear you snicker behind your monitor "this won't happen in Perl because Perl protects you from such stuff!"

You're right, this won't happen in Perl. But here's a Perl example:

At $work I've written a program that runs checks on servers (rest assured, it's not a Nagios replacement) and creates an array of the results of each server that was tested. The array contains timing and whenever there's a timeout, it contains an "undef" to indicate a penalty.

Has Lacuna Expanse been created solely to cripple Perl developers' productivity?

Are tiobe.com the real developers of Lacuna Expanse, in a fiendish plot to further lower perl's ranking?

Who cares?!
The Saben Semesne are attacking!
Where are my spies?!

I'm looking for a Catalyst or other Perl based application framework that will do standard CRUDS (CReate, Update, Delete, Search) functions. I've seen a few of the online writeups about how different people put these together in as few lines as possible, but I'm more looking for one that I can just install via CPAN, configure a database connection, and start the server.

Any suggestions?

(This is a long time overdue post in Ubic series. Previous posts can be found here.)

Last time I explained how any kind of service start/stop/status behavior can be encapsulated in an Ubic::Service subclass. But what if you have many similar services, and want to generate the service list dynamically?

For example, sometimes you already have a directory with daemon configs, and want to run them all without creating additional "service description" file per daemon.
Or you may wish to start ten instances of one processing script.
Or maybe you keep the list of your services in SQL database on another host.

In all these cases, multiservices come to the rescue.
Multiservice is a container object which provides part of the service tree.

I was doing a lot of work on Dancer lately. We've been experiencing a few test failures on Windows with the upcoming big release. I'll admit we are usually reluctant to go to Windows-land and fix issues, but Alias sure made it clear how important it really is, so it was something that had to be done at some point.

While investigating this situation, I've noticed Cwd's realpath() function was complaining a path for a test did not exist. The path did not exist for Linux or BSD either but the test wasn't failing there.

After checking out Cwd more closely, I've noticed it had an XS version for Linux, BSD and other operating systems and a few Pure-Perl implementations of realpath() for some (OS2, MSWin32, DOS, VMS, for example). Upon inspecting the XS code and the Pure-Perl code it was apparent that the versions were not exactly the same. On Windows if a path does not exist, it will croak but on Linux it will continue running.

Blessed are the CPAN Testers, for they show us the bugs!

Playing games can be useful too. While playing "Lacuna Expanse" and using a bot to find a good asteroid to mine, I found a bug in YAML and fixed it. Later I found out that this bug was already reported 4 years ago.

as the next release 0.4.4 is about basic editing, I filles there several feature holes.

like e.g. inserting rectangular selections properly, making the brace nav more consistant (jumping from inside to inside matching braces or from outside to outside). then i also added left and right to moving text(line/block) to indend/dedend tabwise which makes refactoring easier. Now im working on the proper DND of rectangular selections, which isnt delt properly even in kommodo 5.

here on austrian perl workshop the friendly guys helped me to fixe an issue with Module::Install and thanks to jizef kutej the debian package is also ready (jejky!).

but there is much more to come but i want to sanatize the basics first befor we start next bigger chapters like plugins and extentions ....

http://corehackers.perl.org/wiki/index.php5?title=Main_Page#Performance_Hacks lists some perl5 Performance hacks.
Most ideas are from me.

There's another Raw Array idea from Leon, for which I wrote Tie::CArray ages ago, but this can be made better of course. Tie::CArray does not need to do its own memory management.
That's what we have magic for. The array of I32/U32/NV/char[12]/.. can reside in the PVX and the getters/setters/... are just magic callbacks as in Tie::Array.

I just talked about some obvious illguts observations at the vienna perl workshop and was asked to eloborate.

Leave off xpv* structs

If you don't need any xpv struct, e.g. for short strings, you can leave it off. This will need a flag bit and can be set if MAGIC and STASH is empty.

http://cpansearch.perl.org/src/RURBAN/illguts-0.35/index.html#svpv

David Precious (A.K.A. bigpresh) recently wrote a nifty little webapp for MPD (Music Player Daemon) using Dancer, available here.

He also wrote an article about it which you can read here.

Props!

I ran into some trouble when combining tainting with the Encode module. Recently, I've been using the Encode module to decode from binary to text as soon as possible and encode back to binary as late as possible. Unfortunately, this completely kills the protection that -T grants, presumably b/c the Encode module uses a regular expression to do it's work.

#! /usr/bin/perl -T

use strict;
use warnings;
use Encode();

$ENV{'PATH'} = '/bin:/usr/bin';
delete @ENV{'IFS', 'CDPATH', 'ENV', 'BASH_ENV'};

my ($home) = Encode::encode('UTF-8', $ENV{'HOME'});
system("echo $home");

This can be beaten by wrapping calls to the Encode module (and others such as Getopt::Long with calls to the Taint module to make sure that if a variable is tainted before it goes through the Encode module, it stays tainted after it comes out of the Encode module.

However, for me, an unexpected result of combining two great parts of Perl

In my previous post (you should read it if you haven't), I eventually arrived at this:

On Stackoverflow, someone asks how to get 100 random numbers without a loop. It's one of those dumb homework problems that tries to forbid only one of many things instead of specifying the technique it really wants the student to practice. In Perl land, that leaves the door open for the sick and twisted minds of people such as Tom Christiansen and Sinan Ünür. I wonder if the teacher would even understand their solutions, much less accept them.

Many of the answers went to great pains to avoid certain Perl keywords while still creating loops. Some people debated if map is a loop. Some people used recursion, forgetting that every time you recurse in Perl, God kills a kitten. Maybe someday people will realize that recursion in Perl isn't the same kung fu they see in other languages. Perl actually must recurse because it has no way of knowing if it's going to call the same function definition.

Most of the solutions used Perl's built-in rand, which I think ignores half of the problem, the random numbers themselves. I use rand too, but I replace its definition to use the random.org web service to get lists of random numbers generated from atmospheric noise. Not only that, but I change rand in very sick and twisted ways, adding a list context.

I'm doing a lot of jQuery of one of my current clients. Along the way I'm learning fun Javascript gotchas. Here's a demonstration of one scoping difference. Below the .change() event tied to bar works as I expected, but baz is a global variable.

function addARow() {
   var i = $('#next_id').val();
   var d = $("div"),                                      //COMMA then foo
   foo = $("<input>",{id:"foo" + i,size:"3"}).appendTo(d)
   .change(function() { bar.val($(this).val() * 2); })
   .change(function() { baz.val($(this).val() * 2); }),   //COMMA then bar
   bar = $("<input>",{id:"bar" + i,size:"3"}).appendTo(d);//SEMICOLON then baz
   baz = $("<input>",{id:"baz" + i,size:"3"}).appendTo(d);
   $("<br>").appendTo(d);
   $('#next_id').val(i - 1 + 2);   // + 1 would concat 1 onto the string  :)
};

See it in action here. jQuery creates a set of 3 input fields (foo, bar, baz) on each click of "add". When foo is changed by the user, bar and baz are updated. But the wrong baz gets the update.

Live and learn. :)