Rakudo.js update - NFG / unicode collation and role bug fixes

Rakudo.js update - NFG, unicode collation and more bugfixes

Rakudo.js has been in bugfixing mode recently.

Rakudo.js now uses NFG (Normal Grapheme Form) semantics in some places.
This means some string operations treat strings as sequences of graphemes instead of unicode code points. Graphemes are "user-perceived characters" (See http://unicode.org/reports/tr29/). This isn't done everywhere yet but it allows us to pass a bunch of roast tests.
Because JavaScript doesn't use graphemes underneath in it's string implementation like MoarVM does using NFG semantics can be much more expensive.
As such in low level setting code we often want to use the native javascript semantics when they are good enough.
To make that a choice I added a bunch of NFG aware op variants like (nqp::charsnfg) so we can pay the price only when it's necessary.

Running Perl for your car's dashboard display

About time!

Moose Delegates

Today in the Moose-Pen I am going to look at something new 'Native Delegations' on attributes or to say it another less wordy way 'Traits'

Native Delegation is a way to let you treat standard Perl data structure as if they where objects. So to take an example from my test cases say you have this

foreach my $index2  (0..(scalar(@{$predicates[$index]->predicates()})-1)) {
that (scalar(@{$predicates[$index]->predicates()})-1)) is not very readable. Now suppose we could just have this

($predicates [$index]->predicates()->count-1)
a little easier to read. Now one can do this kind of thing in plain Perl but it can be a little code gymnastics if you are really interested in how much code it is check out the 'DBD::_::common' package for an example of a partial work up for a Hash object. You can also have a look here to see how to set this up for an ARRAY or if you have trouble falling asleep. There is even a package called Tie::Array

Perl 6 Core Hacking: QASTalicious

Read this article on Rakudo.Party

Over the past month, I spent some time in Rakudo's QAST land writing a few optimizations, fixing bugs involving warnings, as well as squashing a monster hive of 10 thunk scoping bugs with a single commit. In today's article, we'll go over that last feat in detail, as well as learn what QAST is and how to work with it.


"QAST" stands for "Q" Abstract Syntax Tree. The "Q" is there because it's comes after letter "P", and "P" used to be in "PAST" to stand for "Parrot", the name of an earlier, experimental Perl 6 implementation (or rather, its virtual machine). Let's see what QAST is all about!

Dumping QAST

Every Rakudo Perl 6 program compiles down to a tree of QAST nodes and you can dump that tree if you specify --target=ast or --target=optimize command line option to perl6 when compiling a program or a module:

Little Moose Tries to Run

Well more testing and re-factoring with just a little Moose in the Moose-Pen today. So I spend a good deal of time getting my tests cases all in a row. Now just a few interesting details on that, I created a number of generic utility tests that can be reused in my test cases. What I did was take the 'cmp_deeply' test like this on in 33_constants.t

   my $da_conditions  = $da->conditions();
   my $dad_conditions = $dad->Conditions();
   my $in_conditions  = $in_hash2->{conditions};
   foreach my $index (0..scalar(@{$in_conditions}-1)) {
     my $in   = $in_conditions->[$index];
     cmp_deeply($da_conditions->[$index]->predicates->[0], methods(%{$in}),
"DA predicates correct" );
     cmp_deeply($dad_conditions->[$index]->predicates->[0], methods(%{$in}),
"DAD predicates no $index correct" );
      # cmp_deeply( $dad_conditions->predicates->[$index], methods(%{$in}),DAD predicates correct" );
and converted them into a sub, in this case deep_predicates, then created a packages to hold them ' Test::Database::Accessor::Utils ' and then rather than the above all I call is

Relax it is Just Moose

Well it clean-up and re-factor day in the Moose-pen. So no new Moose Majick today just cleaning up and adding in stuff that I have left out (forgotten actually) and moving things about.

Well the first thing I did was finish off the last three of my remaining Accessor attibutes, 'gathers', 'filters' and 'sort' or if you are SQL inclined 'group by', 'having'. and 'sorts'.

The 'gathers' and 'sorts' attributes are just a pair of 'ArrayRefofElements' which I covered off in this post and types and the 'filters' is a 'ArrayRefofConditions' which I covered in this post. I did of course spend a little time making sure that my DAD role attributes matched up with the accessosr ones and as this is me programming I created two new test cases, 37_gathers.t that handles filters as well, and 39_sorts.t

Low Stress Moose

So today in the Moose-pen I am going to move onto something a little differtn and that is my Accessor 'links' attribute which I am going to cover with the 35_links.t test suite.

I did a quick review of the bits and pieces I had stubbed in so far and I notices I need a few thing. First had a look at my Accessor 'links' attribute and I will need a new type for that. So to get this new 'ArrayRefofLinks' type I just did what we have seen in a number of older posts, namely add in a 'use', a 'class_type' and a 'subtype' into my Types.pm, I will hold off on a coercion for now till I get a little deeper into my post. So that takes case of 'links'.

I also noticed that 'Links' attribute of my DAD now had the wrong type as well so that was change to the 'ArrayRefofLinks' type

Looking at my 'Database::Accessor::Link' class I have never liked this two attributes

Sophisticated Moose

Today in the Moose-pen I am going to play with the very familiar 33_condtions.t and this time I am going to solve another little API problem I know I am going to run into.

Part of my evil plan is to allow end users to add in conditions on the fly and I have made provision for that with the 'rw' 'conditions' attribute found on my DAD Role. Now I have to fix a minor oversight on my part can change the type on 'conditions' to be the same as the original in Accessors, so I changed this

–    isa     => 'ArrayRefofPredicates',
++    isa     => 'ArrayRefofConditions',
What I want to be able to do is allow my end users to do something like this

  my $da = Database::Accessor->new($in_hash2);
  $da->DAD->add_condition({left      =>{name =>'country',
                                        view =>'People'},
                           right     =>{value=>'Slyvania'},
                           operator  =>'=',
                           condition =>'AND'});

Late Night Moose

Well in today's Moose-pen I am still going to play about with good old 33_condtions.t , not the tests per-say but the API they are calling. In my last post I fixed the my API call by adding in the 'predicates' param to the 'conditions' param like this

 view =>'People' } ],
conditions=>[{predicates=>[{left=>{name =>'last_name',
                                   view =>'People'},
to me the above is a little wordy an error prone. The original API call I attempted

      view =>'People' } ],
  conditions=>[{left =>{name =>'last_name',
                        view =>'People'},...
in this post was nicer. Now how to achieve that??

Well I could use 'around BUILDARGS' call like I did for various 'read' and 'write' flags from this post. With the 'around' I could then manipulate what is coming and return what I want. Now the problem with that is I would get no class level re-use from it. Each class that used the same pattern or in this case type of 'ArrayRefofConditions', would require it own 'around' clause.

Call for Presenters TPCiSLC 2018

Talk submissions for #TPCiSLC are currently being accepted! Round 1 closing/Round 2 opening Jan 28/29.

We will be accepting proposals for the following session types:

Short Talks (20 minutes)
Standard Talks (50 minutes)
Tutorial Sessions (80 or 110 minutes)

To submit proposals for a talk/presentation, please fill out this form: https://goo.gl/forms/2qxdfgtRsdc6lXBZ2.

We encourage you to submit your talk as early as possible. We will be choosing speakers in an ongoing process, with the final submissions to be accepted no later than March 18th, 2018.

Round 1: January 1st - January 28th, speakers notified by 2/7/18
Round 2: January 29th - February 25th, speakers notified by 3/7/18
Round 3: February 26th - March 18th, speakers notified by 3/28/18
Final Speaker Lineup Announced: April 1st, 2018

Apply at https://goo.gl/forms/2qxdfgtRsdc6lXBZ2

More information at https://perlconference.us/tpc-2018-slc/cfp/

Not a Mousse Moose!

So in my last Moose-pen I ran into problem with my 33_conditions.t test case where it was giving me errors like this

# Compared $data->left # got : Does not exist …
Well after some playing about I saw the errors of my ways. Looking at the Database::Accessor::Condition class I remembered to load that properly I will have to pass in any conditions with the 'predicates' param. So I modified my$$in_hash to this;

                             right=>{ param=>'test'},
                                  right=>{ param=>'test'},
I ran my test again but then I was back at square one again?
Attribute (name) is required at C:\Dwimperl\perl\site\lib\Moose\Object.pm line 24 Moose::Object::new('Database::Accessor::Element', 'param', 'test') called at D:\GitHub\database-accessor\lib\Database\Accessor\Types.pm line 29
Hmm... So it looks like it does not like my 'param' coercion again. I double checked my 'Database::Accessor::Predicate' class and saw that I had

Just A Cute Moose

Well today in the Moose-pen I am still going to have a look at my 33_conditions.t test case. In my last post I got all the little bits working right up to the point when I want the DAD to return or at least crate this string


After sleeping on it and then playing a bit I came to the very wise conclusion to not to test my Accessor in this way. If I take this path then I am opening myself up to going down all sorts of test rabbit holes where there is nothing wrong with my Accessor but a problem in my Tests DAD. So what to do?

Well lets have a look at what I have to test:

  • coercion from a hash to a type works correctly
  • class attribute values passed into the DAD are correct

Best Moose Opening Line Ever

So today in the Moose-Pen I was going to get my 33_conditions.t test case working. New we left off in my last post with this call to Accessor

my $user = Database::Accessor->new(
        view     => {name  => 'People'},
        elements => [{ name => 'first_name',
                                 view=>'People' },
                             { name => 'last_name',
                                view => 'People' },
                             { name => 'user_id',
                                view =>'People' } ],
        conditions=>[{ left=>{name=>'user_id',
                               { condition=>'AND',
                                 right=>{param=>'John'}},  ]
which in SQL is

SELECT People.first_name,People.last_name, People.user_id 
  FROM People
 WHERE  People.user_id = 'test'
   AND  People.first_name = 'John'
and Mongo something like this

  db.People.find({$and: [{user_id: {$eq: test}},{user_id: {$eq: John}}]},{ street: 1, city: 1, country: 1}
Not I do not have a working Mongo or SQL DAD right not so the above are just for show so I have to use my fudged results from my last post with my Test::DAD. So I am looking to get this


Less Moose more Elk

Well back to clean up mode in the old Moose-pen today as I have completed my first round of changes to my Database::Accessor embedded classes. Just taking a quick peed at what might attributes might be missing from my I see that my Predicate is missing something important at least to any logical predicate. That is the ability to add parentheses to a predicate.

Well the way I see it I can just make this a Boolean flag attributes such as 'parentheses' but that takes a good deal of flexibility from the system. So what I am going to go for is two flags one for open and one from close, like this

   has open_parentheses => (
      is  => 'rw',
      isa => 'Bool',
      default => 0,
      alias    => [qw(open open_paren)]

    has close_ parentheses  => (
      is  => 'rw',
      isa => 'Bool',
      default => 0,
      alias    => [qw(close close_paren)]

Long Live Perl 5!

You probably have read a recent Open Letter to the Perl Community. The letter has generated a lot of response (173 reddit comments as I write this). Unfortunately, a lot of the responses are quite negative and do not match my understanding of the letter.

I figured I share my interpretation of the words and if that interpretation does not match the author's intent, I hope my interpretation captures the mood of the community on the matter.

A Radical Idea

The first thing the letter talks about is what it calls a "radical idea". The suggestion is for Perl 5 compiler to upgrade its backend. Instead of the current Perl VM the perl compiler uses, it would be ported to NQP compiler toolkit and will support all of the backends NQP supports, which currently is MoarVM, as the leading option, with JVM and JavaScript backends at various stages of completion as alternatives.

Moose on the edge

Now in the Moose-pen we are coming to the home stretch of my Database::Accessor Classes, starting with 'Gather'. This one represents a fields in an in SQL 'Group by' clause, and in Mongo 'Group' stage.

Looking at it it really is just an array of predicate clauses so I think I can just drop this class altogether, which is always a good thing. So out it goes and the test as well. So that was easy.

Next it 'Filter' which is a 'Having' clause in SQL and I not 100% sure what it is in Mongo most likely something to do with group and project or maybe match? I guess it is up to the Mongo DAD writer to come up with that one.

Like 'Gather' this is just an array of predicates or the same as my 'Condition' class so I can just drop that one and its tests as well. So that is two classes I do not really need.

The Perl Conference in Salt Lake City, 2018

We are pleased to FINALLY bring you the news everyone has been waiting for!

It is with great enthusiasm that we happily announce The Perl Conference in North America, 2018 will be held Sunday, June 17th through Friday, June 22nd at the Little America Hotel in Salt Lake City, Utah!

The main event will run from Monday, June 18th through Wednesday, June 20th. Master Classes, training, and other activities will be held on the 17th, 21st, and 22nd. We highly encourage you to attend all days, but especially look forward to seeing you at the Main Event.

This year’s website is already live and can be found at https://perlconference.us/tpc-2018-slc/.

Call for Participation is live at https://goo.gl/forms/PkYHWD3B4vUKZLrW2. Please consider submitting a talk!

Registration will also be live February 1st, so watch out for that!

Keep watching for additional announcements.


Can a Moose Juggle?

The next class in the Moose-pen is Database::Accessor::Link which would represent a 'Join' in SQL and a I think they call it a 'Pipe Lookup' in Mongo or it could mean a find on a document buried under the parent document. I guess that is a topic for a later post.

The first thing I noticed was this class and the 'Condtion' class I looked at yesterday is they share the same 'predicates' attribute. So being a good Mooser the first thing to day is to pull that attribute out and add it into a role.

So after a few copy and pastes and a little typing I have

    package Database::Accessor::Roles::PredicateArray;
    BEGIN {
        $Database::Accessor::Roles::PredicateArray = "0.01";
    use Moose::Role;
    use MooseX::Aliases;
    has predicates => (
        traits  => ['Array'],
        is      => 'rw',
        isa     => 'ArrayRefofPredicates',
        coerce  => 1,
        alias   => 'conditions',
        handles => {
            _add_predicate   => 'push',
            count_predicates => 'count',
and in my Condition class I just add in the role

    with qw(Database::Accessor::Roles::Alias
++             Database::Accessor::Roles::PredicateArray

Late Night Moose

In today's Moose-pen we are going to have a look on one of the first my compound Database::Accessor classes 'Database::Accessor::Condition'. By compound I simply mean a class that holds other classes.

The Condition class is just as it sounds a class to add a logical condition to an Accessor. In SQL it would be a 'where' clause and Mongo some of the 'Aggregate' clauses. So basically this is a class that holds a number of 'Predicate' classes. So all I need to add is;

 has predicates => (
        is      => 'rw',
        isa     => 'ArrayRefofPredicates',
        coerce  => 1,
        alias   => 'conditions',
        traits  => ['Array'],
         handles => {
            add_predicate   => 'push',
            count_predicates => 'count',

More Than One?

Well today in the Moose-pen I am going to have a look at my Database::Accessor::Parm. Now this is one of the more use full of my classes, This is the one that is used to supply data to a DB query. So in DBI you may have something like this

 sub some_sql{
     my ($in) = @_;
     my $sql =  'SELECT * FROM person where person.id=?'
     my $sth -= $dbh->prepare($sql);
     my $person =   $sth->execute($in);
The '?' is replaced with the '4' in the execute. Now in my model that '4' would be my 'param' and I would pass it as a scalar value. Now I would hope the person writing my DAD SQL write would do it like above and not like this

sub some_sql{
     my ($in) = @_;
     my $sql =  'SELECT * FROM person where person.id=$in';
     my $sth -= $dbh->prepare($sql);
     my $person =   $sth->execute();
I would not like to see that JAVA SQL-Injection attack

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl and offering the modern features you’ve come to expect in blog platforms, the site is hosted by Dave Cross and Aaron Crane, with a design donated by Six Apart, Ltd.