Looking at DBI Part 17, Wasn't that a Police Album

In part 16 we has a quick look at the NAME and other attributes that can help you create portable code. Well this time lets break with good portable code like I did in part 15 and bend the spec a bit. We will have a look at Asynchronous Queries

Wrapped Around Your Fingers
So we are not talking about the Album but then again I think the album should of been called Asynchronicity, as it really was overplayed not very well executed and over-hyped, anyway lets get on track.

Asynchronous in DBI means we are sending a query off to the DB and instead of waiting for a response, we go off and do our own thing and DBI and hopefully later we can come back and ask if our results are ready.

Planet Moose - November 2013

Welcome to Planet Moose, a brief write up on what's been happening in the world of Moose in the past month, for the benefit of those of you who don't have their eyes permanently glued to the #moose IRC channel, or the MetaCPAN recent uploads page.

If you'd like to contribute some news for next month's issue, you can do so on the wiki.

Mouse 2.0.0

Mouse has rolled over to 2.0.0, and now 2.1.0. 2.0.0 introduced a potentially backwards-incompatible change with regards to role composition, and 2.1.0 drops support for Perl 5.6.x.

GitPrep 1.4 is released - Add private repository and collaborator features

I released GitPrep 1.4 at 2013/10/09. You can install portable GitHub system into Unix / Linux easily. It is second major release.

Because you can install GitPrep into your own server, you can create users and repositories without limit. You can use GitPrep freely because GitPrep is free software. You can also install GitPrep into shared rental server.

GitPrep (Document and Repository)

Features added in 1.4 are:

- Fix markdown small bugs (*em*, _em_ etc)
- Defalt readme file is changed from README to README.md
- Add private repository and collaboration features

Big adding point is that Gitprep support private repository and collaboration features. If you repository set to private, the repository can be shown, pulled, pushed from only myself and collaborator. By this, you can publish private repository on public internet.

Let's use usufully.

Example

You can try GitPrep example.

GitPrep Example

Download

Download

Document

GitPrep Document and Repositry

Why Are Fast Tests Important?

I recently made the following comment on Twitter:

EVERY tech company I've worked with has had a test suite broken in some fundamental way, even if all test pass. No exceptions.

Unfortunately, Twitter is not a great medium for exchanging ideas like this, but when challenged on this by one person, I mentioned:

Code + path coverage? Test suite runtime. Duplicate code. Procedural tests for OO code? Cut-n-paste? Skipped tests? Organization?

There's a huge amount to cover there, but I want to touch on test suite runtime for just a moment.

Looking at DBI Part 16, $STH Comes to the Rescue

In part 15 we had a look at a look at some extra XML goodies that DBD::Oracle give you. Now we are going to see how the statement handle can save you arse.

LOGID, LogID, LOgid

Egad those really are three table names I once encountered in a Oracle DB and I am not even going to show you all the fields in some of these three that where upper, lower and even mixed case.

Well how can DBI statement handle help?? Well you can at least get everything out of a table in the same case. So lets take this SQL for example

Select "First_Name", "last_NAME" from USER_NAMES

Don't laugh I actually had that above table. You could of course just deal with it as you go along but the DBI $sth has the handy 'NAME_lc_hash' attribute which can does the dirty work for you.

So on the select from this table you could do something like this

DWIM Perl for Linux 5.18.1

A long time ago I started to build a Perl distribution for Linux, but then I did not have more time and I did not have the immediate need. But it has changed and thus I created a new version of the DWIM Perl for Linux.

It includes perl 5.18.1, DBI, DBD::SQlite, Moo, and for the first time it include XML::LibXML. This is the big change for me and for the package. It now includes a directory called c/ where external (non-perl) dependencies are installed.

If you have time, please give it a try!

Keeping a Reasonable Number of Incremental Backups

My servers are backed up using a little script that works just like Apple's TimeMachine. Every day a backup job moves the latest backup to a folder named by the date of its creation and creates a new backup in the folder latest. Rsync has a nifty feature that creates hardlinks to files in a different folder if the file matches with the current folder. Meaning that you don't create a full backup every time but only sync the differences, i.e. an incremental backup.

All in all that works quite well and I can go back to any date in time to see the state of the machine. This script is now in production for quite a while and I ended up running out of space because I haven't figured out a good way to get rid of old backups. A listing for one of my servers looks currently like this:

Perl 5 Optimizing Compiler, Part 10: Kickstarter & Performance Benchmarks

Howdy Perl Community,

We've released initial RPerl performance benchmarks, which are very promising indeed!

http://rperl.org/performance_benchmarks.html

Also, our Kickstarter campaign is now live, please consider backing the RPerl v1.0 project, our pledge drive ends on Wednesday 12/4 so don't delay!

http://www.kickstarter.com/projects/wbraswell/perl-5-optimizing-compiler-rperl-v10

Thank you all so much for your support to make Perl better and faster for everyone!

Perling,
~ Will the Chill

Looking at DBI Part 15, Something Extra

In part 14 we had a look at a look at blob, lob and clobs and even memo data sometimes DBD developers add in a few goodies that are outside the spec despite the ever watchful eyes of the DBI gods. Here is one exmple.

XML, That will never catch on we have SML

On neat little trick, besides a bunch of others, that DBD::Oracle can do is suck up large XML documents into its native XMLType.

It is actually quite easy as all that is really necessary is to import the correct data type and then bind it to the appropriate column.

So to start

use DBD::DBD qw(:ora_types);

and then simply bind it with the ORA_XMLTYPE like this

my $sql = "insert into big_data values(:p_xml)"; my $sth =$dbh-> prepare($SQL); $sth-> bind_param(":p_xml", $big_hunk_of_xml, { ora_type => ORA_XMLTYPE }); $sth-> execute();

And that's it.

Let Your Fingers Do the Walking

Inline tests with Dist::Zilla

I'd like to give my debut to the community with a small module whose code I snatched from someone else ;-) - Dist::Zilla::Plugin::Test::Inline.

It's a plugin that integrates Test::Inline into Dist::Zilla by looking for inline (POD) tests and exporting them into *.t files when you build your module.

Hope it's helpful for anyone...

Perl in the news - A public bus service case

It's starts here:

Yep - Slashdot

MooseStyle: Self-made inline vs cpan-based inline


I have read the "Add-Moose-style-attribute-accessors-to-your-Perl-classes" article from David Farrell.


I understand his issue with "sans dependencies" but was uncomfortable with
his solution. To write this code-snippet everytime (or copy&past)
and maintain every file feels wrong.

I rememberd of Mo and Mo:Inline.
What turned out to be better than in my memory.

What are your opinions?

Looking at DBI Part 14, I am the Blob!! not the Blob!

In part 13 we had a look at a look at bind_col now we will take a quick look at blob, lob and clobs and even memo data.

Some Good! Some Bad!

The one thing about memo, blob, lob, clobs and other large DB fields is that not all DBDs use em, or even care about them, but when you need them you have to use them.

So for fun we will just use an example from DBD::Oracle but in general most of the others that do (DBD::DB2, DBD::Sybase) follow the spec fairly closely, One first has to tell the DB what type of data to bind, set up a maximum length to read and or a length to stop reading.

Give it a Try

So the first thing we have to do it tell the bind_param() method what sort of LOB to expect. So your bind call would look like this:

$sth->bind_param(:p_big_desc, $big_desc, {ora_type=>ORA_CLOB});

Or if you are dealing with a BLOB:

$sth->bind_param(:p_pic, $a_pic, {ora_type=>ORA_BLOB});

Treat Criticism and Complaints as Defects

If you have a customer who complains about something, don’t ignore it, no matter how crazy you think the complaint may be. If it can happen once, it can happen again; and more importantly, how many times has it already happened and nobody brought it up? 

Of course not all criticism is valid, but you don’t know if it’s valid if you don’t take it seriously. Investigate every claim. Find the root cause of the complaint just as you would find the cause of a software defect. And once you’ve found it, see changes you could make, or what systems you could put in place to either stop it from happening, or mitigate the damage it could do. 

If you treat every complaint as a defect, you’ll have fewer and fewer complaints over time. In addition, there’s simply nothing better than good customer service to promote word of mouth marketing.

[From my blog.]

Introducing Fetchware: package management for source code distributions.

Years ago when I was a linux noob, I loved to install software from source code instead of lame vendor packages. It was cool. And it followed naturally from using Slackware as my linux distro of choice as Slackware still has more of an old school Unix feel to it instead of easy to use Ubuntu lameness. It was also encouraged by Slackware, which probably has the smallest repository of all linux distros, because it is still limited to what fits on 6 CDs or 1 DVD, but there are Slackbuilds for extra packages. What I loved most was getting to choose how the program was configured. Figuring it out was usually a simple ./configure --help away.

How to resize a NTFS qemu qcow2 image (the easy way)

There's so much bad information out there how to simply resize a NTFS qemu qcow2 (windows in kvm) image, and I need to frequently enhance my windows images, esp. on win8 (64 bit) so I'll document it here for the next time:

  • Don't use fdisk if you have gparted!
  • Don't waste space converting a qcow2 to raw
  • Shutdown the vm!

  • $ sudo su -

  • $ ls /var/lib/libvirt/images/*.qcow2
  • $ qemu-img resize /var/lib/libvirt/images/windows_x64.qcow2 +5GB

  • $ modprobe nbd max_part=63

  • $ qemu-nbd -c /dev/nbd0 /var/lib/libvirt/images/windows_x64.qcow2

  • $ # fdisk -l /dev/nbd0 # or better

  • $ kpartx /dev/nbd0 # to check the partitions and sizes
  • $ ntfsresize --info /dev/nbd0 # to check if the volumes are not dirty

  • $ gparted /dev/nbd0 # enhance the partition size to max

  • $ killall qemu-nbd

  • Start the vm and let windows do the chkdsk /f (in win8 automatically)

  • In winxp you might need to do ntfsresize -x /dev/nbd0p2 manually

Looking at DBI Part 13, An Evil Twin!

In part 12 we had a look at a look at the array_execute and now we are going to look at bind_param's Evil Twin bind_col.

Bind what?

DBI's master chef designed it from the outset for 'SPEED', to get quickly to the point any extra under the hood calls to DBI will slow things down, bind_col eliminates some of the under the hood call to DBI and will make your assignment magically disappear

Lets that this example;

$sql = 'SELECT city, state_province, country_id FROM locations'; $sth = $dbh->prepare($sql); $sth->execute();

while (my $row = $sth->fetchrow_arrayref()){
my $r_city = $row->[0];
my $r_state = $row->[1];
my $r_country = $row->[2];
print "$r_city,$r_state,$r_country";
};

Remembering that is it best to put our bind_col assignment after the execute we change the code to this;

$sql = 'SELECT city, state_province, country_id FROM locations'; $sth = $dbh->prepare($sql); $sth->execute(); my ($r_city, $r_state,$country);

$sth->bind_col(1,$r_city);
$sth->bind_col(2,$r_state);
$sth->bind_col(3,$r_country);

while ( $sth->fetchrow_arrayref()){
print "$r_city,$r_state,$r_country";
};

SOC Agile Development

Update: It's been modified slightly. Read the official announcement and the follow-up instead of this post.

As some of you know, one of my most popular talks is "Agile Companies Go P.O.P.". What you probably don't know is that this talk is actually an introduction to a full day training course I give on this topic. So far it's been a lot of fun and I cover everything from the space shuttle (which you've seen from the slides), to the Feudal era (which you haven't). However, there is one topic I don't cover enough and I think it's time to point it out: agile sucks, but it sucks less than the alternatives.

Is it possible to run Movable Type under Apache HTTP Server?

That was the question posted to Quora which seemed strange to me given that Apache is the web server most used by MT end users, developers and even (to my knowledge) every single core developer who has even worked on MT. Not to mention the rest of the world! I'd love to know what spurred on the question and what web server people generally use in the author's world.

Anyhow, the reason I'm writing is mainly because the REST of my answer which is a rant about still-persistent use of CGI by some administrators/developers. facepalm

My answer, after the jump, just in case Quora goes the the bit graveyard one day...

Simplifying imports (mainly in tests)

During test writing often I find myself having to use a lot of modules and setting up various things that should be common for all of my tests in a given project. In some examples it may get ugly and I really don't like having a ~30 lines long setup in every single test file of mine. After some experimenting, I came up with a quick-and-dirty solution. I create a module in my 't' directory (let's say t/lib/Test/MyApp.pm) and put everything I need into it's import sub:

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl with a graphic design donated by Six Apart, Ltd.