Perl QA Hackathon report - part 3: Net::SSH::Perl v2

This year at the Perl QA Hackathon I had three topics: benchmark update Perl until 5.24, enable CPAN test reporting on L4Linux, release a Net::SSH::Perl v2 to CPAN.

Part 3 - Net::SSH::Perl

I am co-maintaining Net::SSH::Perl, though usually I just apply patches that come up on RT or github.

Some months ago Lance Kinley implemented modern ciphers like AES, more key exchange algorithms, etc. on github - however he started from a CPAN .tgz snapshot. With the help of E. Choroba I got Lance's history rebased to my repository. However, I was short on time and wanted to do release polishing during the hackathon, and so I did.

The only trouble I had was that another patch from Brad Lhotsky which I merged earlier did conflict with Lance's changes in a way that I could solve in a git way but some tests kept failing and I did not understand why.

To be sure to not screw things up I had to give up merging and only released Lance's extensions to CPAN as a new major release v2.01.

As a side effect I also uploaded another new module from Lance, Crypt::OpenBSD::Blowfish, to CPAN.

If you are a user of Net::SSH::Perl please test if it works.

Well that's not it!!

So lets see how I am going to move those annoying 'sub SQL' that I have in my DA::View and DA::Element packages out of them and into my LSDs.

So going blindly where I have gone before though the I might Role for the Element class in DA::LSD::SQL and see what I can come up with. So I duly added this

Perl 6: Comb It!

In Perl 5, I always appreciated the convenience of constructs like these two:

my @things = $text =~ /thing/g;
my %things = $text =~ /(key)...(value)/g;

You take some nice, predictable text, pop a regex next to it, and BOOM! You get a nice list of things or a pretty hash. Magical!

There are some similarities to this construct in Perl 6, but if you're a new programmer, with Perl 5 background, there might be some confusion. First, using several captures doesn't result in nice hashes right off the bat. Second, you don't get strings, you get Match objects.

While Matches are fine, let's look at a tool more suited for the job: The comb

20160425-Perl6-Comb-It.jpg

Plain 'Ol Characters

You can use comb as a subroutine or as a method. In its basic form, comb simply breaks up strings into characters:

Debugging adventure - why it's hard to estimate

This is the second instance in recent time where I thought it was a quick fix but ended up taking way more time than it should. The first was an iOS app submission which was something Apple changed recently. This is a perl example so posting it here.

Mandrill is shutting down . I was using their service for my domains table tennis match and Brainturk brain games online . I looked at the alternatives and settled with sparkpost . This is the sparkpost example for using perl . It seems simple enough so I asked my employee who is learning perl to try it out . When we ran the example we started getting an error . This example uses Net::SMTP and running it in debug mode gave us this

RCPT TO: <x@mydomain.com>
550 5.7.1 relaying denied

Perl QA Hackathon report - part 2: CPAN testing on L4 Linux

This year at the Perl QA Hackathon I had three topics: benchmark update Perl until 5.24, enable CPAN test reporting on L4Linux, release a Net::SSH::Perl v2 to CPAN.

Part 2 - CPAN testing on L4 Linux

To extend the diversity of platforms on CPAN TESTERS, I brought a laptop with me which runs on the L4Re micro-kernel in order to set up CPAN::Reporter tools on it. The laptop runs Ubuntu 16.04 with the kernel exchanged by L4Linux v4.4.

The only hickup I had was that the information about the operating system kernel is not picked up at runtime of CPAN installation or reporting but taken from the Perl's $Config entry. Once I realized that, I recompiled the Perl, currently using 5.22.1, re-iterated the setup, and let it run during the hackathon, with just occasional reviewing to install missing external dependencies.

Thumbnail image for cpan-reporter-log2.jpg

So if you spot a kernel version looking like 4.4.0-l4-g2be3f0e like in here - that's my L4Linux CPAN test box.

And now for Tea

Now in my last post I left off with a bit of a code mess. Roles bleeding code into each other and my API in a dreadful state.

Well on thing I have seen before in non moose apps was use something called 'Class::Rebless' to rip the 'DA::LSDs' from my DA class but that is sort of an abuse of that code as it really is just for renaming name-space and I am not sure it will do what I want.

I had a look about the Moose::MOP class and found something called 'rebless_instance' which should take my original class, in this case DA with one or more LSD roles back to just a DA class.

Seems like a good idea but I think I would be opening myself up to a whole new slew of problems as I would only have a base object perhaps with it attributes in place and any other roles I may want removed.

So what to do??

A pumpking is born

As some of you might have heard, Ricardo Signes had stepped down from the role of Perl Pumpking. He passed on that role to me.

I have written the following message to Perl 5 Porters, and I think it seems apt to provide it at large to the community here:

("group" in this context is the Perl 5 core development group: p5p.)

Testing with AppVeyor

If you ever wanted to test your CPAN modules on Windows systems without having an own Windows system setup, then you should take a look at AppVeyor. Basically it's something like travis-ci, but just running on Windows. Configuration happens in a YAML file named appveyor.yml.

A sample appveyor.yml file for testing CPAN distributions may look like this:

Overview of current maintainer fails

schmorp recently complained about a maintainer fail in Math::BigFloat, and since this is not an isolated incident of the last years, I try to give a summary of all the maintainer fails of important modules lately, as I happen to fix them privately.

Problems in core ext/ and dist/ modules, maintained by p5p:

These are all doomed modules, as p5p refuses to take bug and security reports, and most of them are not maintained and fixed and improved in cperl only.

Math::BigFloat

Maintained by p5p, responsible: pjacklam

regression in blsflt (left shift) and refusal to fix it, lack of understanding of float math.

fixed in cperl only. in cperl we also heavily modernized all the bigint/bignum/bigrat/Big::* modules to use typed signatures.

there were several podcheck fails, which were promptly fixed. so they do react fast, their only problem is the lack of understanding.

Storable

maintained by p5p, responsible: nobody

Never Role Twice!

Well did a little more playing about with my applied roles on my DA.pm and added in a bunch more testing to see if things where working for me.

First I wanted to see if using a role twice caused any ill effects so I added

These benchmarks seem wrong...

Back in the fall of 2013 I began working on a project called C::Blocks. After some very long detours the project is finally coming to fruition. I recently took it for a spin on a benchmark from the benchmarksgame. The results? Let's just say I was very surprised.

A very stupid, over-clever scoping-based importing trick

In some code I’m working on, I use a module which exposes a whole bunch of package variables as part of its public interface. It does not export them, however. I could refer to them directly by prefixing them all with Some::Module::, but that would be ugly and verbose. It’s also unsafe – the vars stricture will not help you catch typos if you use fully qualified names.

The obvious solution would be to emulate what exporting does:

Perl QAH and MetaCPAN

This year was my first Perl Quality Assurance Hackathon, and even then I could only make 2 of the 4 days. I now wish I'd been to everyone ever, and for the full time!

I've been working on the MetaCPAN project for over 4 years, taking on the puppet master/sysadmin/development VM builder type role, even though that's not really my day job. So after all this time to actually meet Olaf Alders and the recently joined Mickey was a great pleasure:

Many Sides of Moose!!

In my last post I did managed to make a little code progress but I still had one major goal to accomplish and that is to make my Data Accessor smart enough that is would know what Driver to use form the connection passed in.

Well I am glad to say I am now able to do that with this test 02_base.t

ok 1 - use DA;
ok 2 - use DA::View;
ok 3 - use DA::Element;
ok 4 - Person is a View
ok 5 - Street is an Element
ok 6 - County is an Element
ok 7 - City is an Element
ok 8 - Address is a DA
ok 9 - SQL correct
ok 10 - Mongo Query correct

Moose is anything but inflexible and with it I was able to come up with a working prototype very quickly just by moving a few things about and reversing my roles.

So before I had DA.pm as a role. I change that to a class

Virtual Spring Cleaning (part 7 of X) Wherein I Enter the Archives

Originally written for providing installable themes for my photo publishing app, App-imagestream, I wrote two modules that make it very easy to provide a basic theme for a web application in an archive. This still allows for quick customization by overlaying the contents of the archive with the contents of another archive or a directory in the file system.

Storable security problems and overlarge data

Some people might be aware that we (not p5p) fixed Storable already for the most egregious security problems with Storable, with public CVE, explanation and metasploit module, but those bugs are still in Storable as on CPAN and in the latest perl5.24.

Now a new problem arose on the horizon, that p5p dares to fix another bug, the inability to store overlarge strings. Which would be okay if they would have taken our fixes, which fixes the security problem and all overlarge data >I32, which are arrays and hashes also. But they didn't, so I have to complain again. People apparently don't like me to complain about p5p mistakes, but you should be concerned what p5p is doing.

See perl #127743 for the wrong fix, adding 2 new ops, where only one is needed, and this fix only fixes a third of the overlarge problem, and not the security problems. There's no need for them to deviate from the API with a worse fix.

O pumpking! My pumpking!

As many people know by now, Ricardo Signes recently announced that he will be standing down as pumpking once Perl 5.24.0 is released, after four and a half years in the role — not to mention an unprecedented five stable 5.x.0 releases of Perl!

Since the Perl QA Hackathon is the first Perl event Rik has attended since his announcement, we thought it would be fitting to offer Rik a token of our appreciation for the remarkable and tireless work he’s put in during his service. So we closed up the second day of the hackathon with a short presentation and a small expression of our gratitude (and hopefully one that Rik didn’t find too embarrassing!)

In particular, Rik has now joined the very select group of people who’ve received a Silver Camel.

P4220270.jpg

Just a Quick one!

So spent a little time working on my rough code from my last post and made up a working copy that you can have a look at here.

It was easy enough to get the SQL part working, after all I have been using such code my Data Accessor for many years now. It also took only a few minutes create a new DA::Mongo class with and an '_execute' function to come up with the query for Mongo.

With my limited test I get the correct query in SQL
 
SELECT street, city, country FROM person  AS me 
and Mongo as well
 
db.person.find({},{ street: 1, city: 1, country: 1}
So I have accomplished one of my goals; To have the same set of params to my API come up with the correct query in either SQL or Mongo.

Perl QA Hackathon report - part 1: Perl::Formance

This year at the Perl QA Hackathon I had three topics: benchmark update Perl until 5.24, enable CPAN test reporting on L4Linux, release a Net::SSH::Perl v2 to CPAN.

Part 1 - Benchmark::Perl::Formance

To keep a benchmark stable but still allow further development, last year I started to create separate bundles of existing benchmarks, starting with the "PerlStone2015" suite. Once settled I would only touch it for maintenance, and for newer developments I can fork it into an independent "PerlStone2016" suite where I could adapt the timings for newer hardware, other benchmarks, or particular language features.

This hackathon I reviewed and polished it to take reasonable runtime in "normal" mode so it does not take weeks to execute and also in "fastmode" where a benchmark produces results within 1-2 seconds.

The removal of the lexical topic feature in 5.24

Now that p5p removed the broken lexical topic feature in 5.24 it's time to look back how that happened and why it should not have happened, and how cperl fixed that problem.

In 2013 Rafael Garcia-Suarez posted to p5p a plea to Salvaging lexical $_ from deprecation arguing that the recent deprecation of my $_ is wrong, and can easily be fixed.

He explained the problems pretty nicely on the language level, but forgot to explain the internal advantages of a lexical $_ over a global or localized $_. To add that it's pretty simple: A lexical $_ is an indexed slot on the pad array, calculated at compile-time, whilst a local or global $_ is a hash entry in the namespace. It's much faster to lookup, write and restore. The slot index is compiled and carried around in the op. That's why it's called lexical, it is resolved at compile-time, and not at run-time.

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl with a graphic design donated by Six Apart, Ltd.