blogs.perl.org

For the past decade or more, perldoc.perl.org has been a useful and convenient resource for viewing perl documentation online. However, it has suffered from lack of maintenance and mounting unfixed issues over the past few years. Being familiar with the excellent Mojolicious documentation site and how it also can display core perldocs, I reasoned that such features would be simple to provide in this modern framework. And so, what would become perldoc.pl (thanks to a domain acquired by pink_mist) was born.

Its sort day here in the Moose-Pen.

Well I have been on a role since I started on 'Group By' so I figure I might as well get the last of the base SQL code snippets in place. The last one is 'sort' or in SQL 'ORDER BY' and why change something that isn't broke here is a new test case ' 60_order_by.t ' and here it the hash I am going to start with;

my $tests = [{
    key  =>'sorts',
    sorts => [
             {name => 'last_name',
              view => 'people'
            },
            {
              name => 'first_name',
              view => 'people'
            },
            ],
    caption => "Order by ",
    sql     => "SELECT people.first_name, people.last_name, people.user_id FROM people ORDER BY people.last_name, people.first_name",
}];

The first thing I will have to do it fix up my test so I will not check for any prams as this SQL has none and this quick one line fix should handle that case;

    cmp_deeply( $da->result()->params, $opts->{params},
        $opts->{caption} . " params correct" )
      if (exists($opts->{params}));

Next the call to the clause in the execture;

...
    $sql .= $self->_group_by_clause();
++    $sql .= $self->_order_by_clause();
…

It another gather day here in the Moose-Pen.

Now that I did the re-factoring for gather it is time to move on the Driver::DBI and do to code to generate the SQL. Of course I always start with a test case and this time it is a new one 50_having.t. It is much the same as the other test cases so no need to repeat that here, the only important bit is this hash;

my $tests = [{
    index=>0,
    key  =>'gather',
    gather =>{
        elements => [
            {
                name => 'first_name',
            },
            {
                name => 'last_name',
                view => 'people'
            },
            {
                name => 'user_id',
                view => 'people'
            }
        ],
        conditions => [
            {
                left => {
                    name => 'last_name',
                },
                right             => { value => 'Bloggings' },
                operator          => '=',
            },
        ]
      },
    caption => "Group bu with 1 param",
    sql     => "SELECT people.first_name, people.last_name, people.user_id  FROM people GROUP BY people.first_name, people.last_name, people.user_id HAVING people.last_name = ?",
params  => ['Bloggings']
}];

The fist thing I need to do is add in a call to generate the SQL;

...
    $sql .= $self->_where_clause();
++    $sql .= $self->_group_by_clause();
...

Many people moved from GitHub to GitLab after annoucing that microsoft bought GitHub. So did I.

After some initial problems with the settling in, it works quite well. GitLab and the integrated CI is very useful to automatically upload your projects to CPAN if all tests are positive.

The steps are quite simple:

1.) Start with some project on gitlab. It doesn’t matter which one.

2.) On gitlab you can configure the some ci settings in your project

• https://gitlab.com/profile/Your-Project/settings/ci_cd
• in Variables you’ve to insert:
• • PAUSE_USER
• • PAUSE_PASSWORD
    Why here? Because you don’t want to shour your username and passwort in the build log.
    These variable are available in the CI-Job

3.) Add .gitlab-ci.yml like this example:

Its gather in the dream day here in the Moose-pen

Well going to skip a post on creating a gaggle of new tests today and instead do some core coding in Driver::DBI and Database::Accessor. Now believe it or not I had a dream last night that something was wrong in the way I set the architecture of Grathers and Filters, GROUP BY and , HAVING in SQL, in Database::Accessor. Boy this does sound corny!. So today I started to look at that part of my code.

In the original Data::Accessor a 'GROUP BY' and 'HAVING' looked like this

  {    table  => { name => 'people },
        fields => [ {name => 'first_name', } ],
        group_by =>[{name=>'first_name'},
                               {name=>'last_name'}, ],
        having => [{ field    => { name => 'last_name' }, param => 'Blog',}],
   }

and in my new incarnation it aped it to be much the same like this

  {    view  => { name => 'people' },
        elements => [ {name => 'first_name', } ],
        gather =>[{name=>'first_name'},
                        {name=>'last_name'}],
        filter => [{ left  => { name => 'last_name' }, right=>{value => 'Blog',}],
   }

expecting and SQL like this

SELECT people.first_name FROM people GROUP BY people.first_name,people.last_name HAVING people.last_name = ?

My latest Ocean of Awareness blog post is "Parsing Left Recursions".

"A lot has been written about parsing left recursion. Unfortunately, much of it simply adds to the mystery. In this post, I hope to frame the subject clearly and briefly."

I ran across this the other day while writing some sample code for the next chapter of Testing Strategies for Modern Perl.

How can we use long lists of symbols from an imported package and still keep the code readable?

I usually prefer use statements of the form:

use My::Module qw(symbol1 symbol2 symbol3);

Except for specially understood modules, like Moose and Test::More, I don't like to just import everything. Rather I like to explicitly call out only the specific symbols I need.

But what if you need to:

use My::Module qw(
    symbol1 symbol2 symbol3 symbol4 symbol5 etc and so many symbols
    that it takes up several lines all the time in every package
    that uses it
);

There are a few alternative approaches.

Its link day here in the Moose-Pen

Well seeing as my elements/fields and conditions/Where are working quite well I think it is time to move on to the next attribute in Accessor.pm Links.

In SQL joins are the same as Database::Accessor Links so given this SQL;

SELECT people.first_name, people.last_name, people.id, address.street FROM people LEFT JOIN address ON people.id = address.user_id WHERE people.first_name = ?

the

LEFT JOIN address ON people.id = address.user_id

is represented by;

    links => [{type=>'LEFT',
               to  =>{name=>'address'},
               predicates=>[{left=>{name=>'id'},
                            right=>{name=>'user_id',
                                    view=>'address'}
                           }]}]

With upcoming version 0.2.0 swat removes usage of prove as internal test runner. There are some -minor- breaking changes due to this. For those who uses swat I would recommend to read GH pages docs and in case you'll need help with migration of your project to the latest swat version don't hesitate to contact me.

Regards

Alexey Melezhik

Read this article on Rakudo.Party

If I were to pick the most ubiquitous construct in the Perl 6 programming language, it'd most definitely be the colonpair. Hash constructors, named arguments and parameters, adverbs, and regex modifiers—all involve the colonpair. It's not surprising that with such breadth there would be many shortcuts when it comes to constructing colonpairs.

Today, we'll learn about all of those! Doing so will have us looking at the simplest as well as some of the more advanced language constructs, so if parts of this article make you scratch your head, don't worry—you don't have to learn all of it at once!

PART I: Creation

Colonwhaaaa?

The colonpair gets its name from (usually) being a Pair object constructor and (usually) having a colon in it. Here are some examples of colonpairs:

:foo,
:$bar,
:meow<moo>,
heh => hah

The last one doesn't have a colon in it, but since it's basically the same thing as other colonpairs, I personally consider it a colonpair as well.

Its where day in the Moos-Pen

Today I am going to expand on the '20_where_basic.t' test case going from a simple one level Element to Param 'where clause' such as

SELECT people.first_name, people.last_name, people.user_id FROM people WHERE ( people.first_name = ? AND people.last_name = ?

to the much more complex Function, Expression, nested and mixed clauses that I was using in the last few posts. First I decided to re-factor my tests from the start and using the 'element_sql_ok' sub from this post I re-factored it to this sub

sub sql_param_ok {
    my $self = shift;
    my ( $dbh, $in_hash, $opts ) = @_;
    $in_hash->{ $opts->{key} }->[ $opts->{index} ] = $opts->{ $opts->{key} };
    my $da = Database::Accessor->new($in_hash);
    $da->retrieve($dbh);
    ok(
        $da->result()->query() eq $opts->{sql},
        $opts->{caption} . " SQL correct"
    );
    cmp_deeply( $da->result()->params, $opts->{params},
        $opts->{caption} . " params correct" );
}

"Marpa and combinator parsing" -- the Marpa algorithm as the basis of better combinator parsing.

This is the second post of a pair on my Ocean of Awareness blog. (The first one was "Marpa and procedural parsing")

This document is the June, 2018 progress report for The Perl Foundation's Perl 6 Constant and Rationals Grant.

---

Tangibles

The bonus deliverable "Perl 6 Numerics" Language documentation page was merged to master. It describes all of the available Perl 6 numerics, their interactions, suitability, and hierarchy.

The bulk of work on constants also has been merged to post-release-2018.06 branch, which will be merged to master after this month's release. I wrote 200 spec tests, available in S04-declarations/constant-6.d.t spec file, and about 500 words of documentation to cover this work.

Rationals

Since my last report, I first continued working on Rationals, focusing on three pieces of work that currently reside in car-grant-unreduce branch

1. Fixing the rare data race and doing some optimizations
2. Fixing bad math in some ops with Zero-Denominator Rationals (ZDRs)
3. Attempting a trial implementation where ZDRs are marked with a role, allowing us to improve performance of some operators.

The newest post on the "Ocean of Awareness" blog is "Marpa and procedural parsing" : Marpa's procedural parsing is more flexible and more powerful than recursive descent's.

Just another quick test postette day here in the Moose Pen.

As we seen in this post I found a bug where I was not coercing correctly on deeply nested Element Attributes. So that means back into the test suite of Database::Accessor I go to cover that bug and the new recusion functionality I have added recently.

Checking the current state of the test suite I see that I never did go deeper than one level of elements, though I do check to ensure that the View on elements are correct and the correct Class is passed in. Howeve I only test for Param and Element classes now I have at lest two others to check for.

I decided it is best to go for broke and create one have that will cover I think all of the combinations I could think of. Here is the hash;

Its re-factor test postette day in the Moose-Pen

Over the past few days I have made up quite a few tests and they where all the same pattern

• Base hash
• add function/expression element
• create DA
• run select/retrieve
• check SQL
• check parms

so a little re-factoring is in order me thinks., Though there is a school of thought that it is a waste of resources to re-factor working test. Fortunately I do not ascribe to that thought so here we go. The re-factoring was quite simple, I took my test;

$in_hash->{elements}->[1] = { function => 'substr',
                                 left  => { name => 'username' },
                                 right => [{ param =>3},{ param =>5}] };
my $da = Database::Accessor->new($in_hash);
$da->retrieve( $utils->connect() );
ok(
    $da->result()->query() eq
      "SELECT user.username, substr(user.username,?,?), user.address FROM user WHERE user.username = ?",
      "Function with 2 param binds SQL correct"
);
cmp_deeply(
           $da->result()->params,
           [3,5,'Bill'],
           "Function params correct"
          ); 

Its get fancy day here in the Moose-Pen

Now that I have 'Functions' nicely working I am going to move on to the next logical field and that is an 'Expression'. In SQL (and 99.9% of other languages) an expression is just a predicate that can be evaluated. So an SQL like this

SELECT user.username, user.salary + 10 FROM users WHERE username='BOB'

In Database::Accessor I have an Expression class for this and it works 90% the same as the Function class I just finished. Thus this expression;

user.salary + 10

is expressed as the following attributes

          +------------+-------------+
          | Attribute  |  Value      |
          +------------+-------------+
          | expression | +           |
          +------------+-------------+
          | left       | user.salary |
          +------------+-------------+
          | right      | 10          |
          +------------+-------------+

in an Expression class and would look like this when used in a DA

         { expression => '+',
                left  => { name => 'salary' },
                right => { param =>10} }, }

As 'Expression' works mostly the same as 'Function' all I really have to do is give the 'Expression' class the 'Element' role and extend the coverage of the 'check_view' sub to include an 'expression' class;

It carry on with recursion day here in the Moose-Pen

Carrying on from where I left off in yesterday's post I was about to add in this new test;

$in_hash->{elements}->[0] = { function => 'substr',
                                 left  => { name => 'username' },
                                 right => [{ param =>3},{ param =>5}] };
my $da = Database::Accessor->new($in_hash);
$da->retrieve( $utils->connect() );
ok(
    $da->result()->query() eq
      "SELECT user.username, substr(user.username,?,?), user.address FROM user WHERE user.username = ?",
      "Function with 2 params bind SQL correct"
);
cmp_deeply(
           $da->result()->params,
           [3,5,'Bill'],
           "Function params correct"
          );

This time round I am testing to see if my new code to handle a 'Function' element will handle a function with more than one option. In this case 'substr(user.username,3,5). Once I got the test in there I gave it a try and got;

Can't call method "view" on unblessed reference at database-accessor-driver-dbi\lib/Database/Accessor/Driver/DBI.pm line 436.

So no free pass today. After a little debugging I traced it back to these lines

Being lazy, I thought to myself, "when using the web interface, why should I need to add my country to the query?". So, using CGI::Lingua which is already available via the VWF system used to build the site, you longer need to do that. From the US, try this:

curl 'https://geocode.nigelhorne.com/cgi-bin/page.fcgi?page=query&q=Rock+Bottom,+Bethesda,+MD'

With VWF it was a minor change:

$rc = $geocoder->geocode(location => $q);

if(!defined($rc)) {

if(my $country = $self->{_lingua}->country()) {

$rc = $geocoder->geocode(location => "$q, $country");

}

return '{}' if(!defined($rc));

}

Unit tests are great. They show that the code was actually designed to a given spec.

Runtime assertions are great. They show that the code is actually running the way it was designed when applied to real data.

Design by contract is a great concept, but it's a bit of an overkill for most projects.

However, sometimes I feel an urge to just rip several lines from a unit test and put them right into production code. Test::More doesn't help here much since my application isn't really meant to output TAP or run in a harness.

So I started out Assert::Refute to narrow the gap:

 # somewhere in production code
 use Assert::Refute qw(:all), { on_fail => 'carp' };

 # in the middle of a large sub
 refute_these {
      isa_ok $some_object, "My::Type", "Correct type detection after decode_json"; 
      is $fee + $price, $total, "Payment parts match";
      like $string, qr/f?o?r?m?a?t?/, "Output is really what we expect it to be";
 };