perldoc.pl now powered by Elasticsearch

perldoc.pl, the alternative perldoc browser, originally leveraged PostgreSQL full text searching. But as I encountered new issues as well as ones I encountered before, I started working to make the search backend swappable.

There are now three search backends available: PostgreSQL, Elasticsearch, and SQLite using the FTS5 extension. While SQLite is of course the simplest to deploy as it requires no setup, I decided against using it for the main perldoc.pl instance as it does not support skipping stopwords, but the backend is now provided so basic search features can be added to a deployment without setting up a database. Additionally, the application can be deployed without any search backend configured, which will just remove the search box and allow viewing of all pages normally.

Little Moose Bug

It is quick fix postette day here in the Moose-Pen

Yesterday I left off with one fail in my '30_where_basic.t' test case

# Expected SQL--> SELECT people.first_name First, people.last_name Last,
 people.user_id "User ID"
# Generated SQL-> SELECT people.first_name, people.last_name, people.user_id 
FROM people
my alias on these Elements

    [
            {
                name => 'first_name',
                alias=> 'First',
            },
            {
                name => 'last_name',
                view => 'people',
                alias=> 'Last'
            },
            {
                name => 'user_id',
                view => 'people',
                alias=> 'User ID'
            }
        ],
are not being adding into my 'SELECT' SQL. This to me was a little odd as I though I had this covered in the '15_alias.t' test case which check for an SQL like this;

'SELECT sys_users.last_name "Last Name", sys_users.first_name "First Name" FROM people sys_users'
I wonder where the bug is. Using that little trick from the other day;

$utils->sql_param_ok( $in_hash,[ $tests->[8]] );
I isolated my testing to the last test re-ran the test and my error was

Iridium Flare End-Of-Life II

Late last year, I blogged about my plans for handling the Iridium flare functionality in the Astro-satpass distribution given that these satellites are being taken out of service in favor of a new design that does not flare.

For some time now, most of the updates to Astro-satpass have been to maintain the canned Iridium status table. This seemed wrong to me, so I have pulled Astro::Coord::ECI::TLE::Iridium out into its own distribution. This is currently available on CPAN. The first distribution of Astro-satpass without this module will be version 0.100, which should be out in a few days.

For people who use one of the CPAN clients, this should be a non-event. I know of no downstream packagers except for ActiveState and MacPorts. Last I looked, ActiveState was just automatically picking up all of CPAN (or not, as the case may be), though if there was no PPM package a CPAN client would work. I have filed a bug with MacPorts describing what is going on. We will see what happens.

Moose Works Harder

Its move along day here in the Moose-Pen

Today I managed to get the time to create '25_exe_array.t' which tests DBI execute array and it went quite well. No real story on that one just with my new 'Test::Utils::sql_param_ok' function I was able to quickly expand it out from just 3 tests to 6 in a few minutes.

I then moved on to '30_where_basic.t' where the idea was to test the basic parts of a 'condition', The basic parts being simple single level predicates mixing elements, functions and expressions.

After creating some 64 tests and giving it a few goes to get rid of the usual typos I ended up with 24 fails out of the 64 and all of them where much like this example;

# Expected SQL--> DELETE FROM people WHERE people.first_name != ? AND 
people.bonus * ? <= ? AND left(people.first_name, ?) != ?
# Generated SQL-> DELETE FROM people WHERE first_name != ? AND bonus * ? 
<= ? AND left(first_name,?) != ?

opensource.com: Confessions of a recovering Perl hacker

"I always insisted on commenting my Perl. I never got to the very end of the Camel Book. Not in one reading, anyway. I never experimented with the darker side-effects; three or four separate operations per line was always enough for me. Over time, as my responsibilities moved more to programming, I cut back on the sysadmin tasks. Of course, that didn't stop the Perl use completely--it's amazing how often you can find an excuse to automate a task and how often Perl is the answer. But it reduced my Perl to manageable levels, levels that didn't affect my day-to-day functioning."

Read on: https://opensource.com/article/18/7/confessions-recovering-perl-hacker

Little Moose Error

Its little bird day here in the Moose-Pen

How does that 'The Be Good Tanyas' song go?

And I sing, the littlest birds sing the prettiest songs …
For us Perl types it should be
And I swear, the littlest typos make the ugliest bugs ...
You will remember that I left off yesterday with this little bug;

Expected SQL--> SELECT people.first_name, people.last_name, people.bonus * abs(?) FROM people
# Generated SQL-> SELECT people.first_name, people.last_name, people.bonus * abs(?),1 FROM people
I was getting ',1' tacked onto the end of some of my generated SQL.

This one proved rather frustrating to track down, and it took much more time than one would expect. I suspected it has something to do with the 'function' part of the '_field_sql' code so I started there but despite my best efforts of adding warnings on each line I could not find it.

How much is too much Moose Testing

Its test test and test again day here in the Moose-pen

Today I added in over 70+ tests into '20_fields.t' and I went though almost every conflagration of fields, params, expression and functions I could think of even the expression from hell from this post.

I did today's work by the book meaning I did all the tests first then I did the debugging. Now I am not going to dump 450 plus lines of hash key value pairs here as that would really piss a few people off and I might loose a reader of two, as if anyone really reads this anyway.

I will just give you what I ran into today. First there was the same bug as yesterday only this time on 'Functions'

Can't locate object method "alias" via package "Database::Accessor::Function" at D:\GitHub\database-accessor-driver-dbi\lib/Database/Accessor/Driver/DBI.pm line 412.

Release SPVM 0.0359

Release SPVM 0.0359

SPVM - Fast array and numeric operation, and provide easy way to C/C++ Binding

0.0359 2018-07-16
  - SPVM::CORE become done native compile
  - add join function
  - fix const assignment bug
  - support list syntax
      my $nums = [(1, 2), (3, 4), (5, 6)];
  - object have body field at offset 0. This will fix alignment bugs.

More Generic Moose

It is putter about some more day here in the Moose-Pen

I was puttering about today with my tests and I discovered something funny on some implications of SQL and it is to do with that 'AS' when playing with 'alias'. Now I knew it was an optional 'key-word' according to the SQL standard what I did not know is a number of DBs will syntax error when an optional key-word is used. So this SQL
SELECT sys_users.last_name AS last FROM people AS sys_users

is correct to standard there are a number of DBs out there that only this will work

SELECT sys_users.last_name last FROM people sys_users

That makes the above one almost 100% universal that I can tell. I originally wanted that 'AS' in there to make the generated SQL much more readable but I have come to realize that '99.9995%' of the time the end user Database::Accessor will never care what the end SQL is just that it works.

New githook-perltidy feature: README from POD

It has been six years since I last mentioned anything about githook-perltidy, a tool for the automatic tidying of Perl and POD files during a Git commit. I rely on it every day, and I still make minor improvements to it, so I thought it worth a quick shout out to others who haven't heard about it or upgraded for a while.

Some CPAN distribution authoring tools come with automatic README generation support. At least in the case of Module::Install::ReadmeFromPod and the various Dist::Zilla plugins I've seen, that generation occurs when Makefile.PL is run. For reasons related to my workflow, Git and GitHub, that timing is too late for me. So I've added a README generation feature to githook-perltidy. See the documentation for details.

One final thing to note. This latest round of work on githook-perltidy was triggered by an unrelated minor issue that a user raised. So don't hesitate to let the author of a tool know when you use something, and that you find it sub-optimal. It might just motivate them to do a bunch of work.

Very Short Moose

Its fix a but post-ette here in the Moose-Pen.

Yesterday I left off with this error;

Can't locate object method "alias" via package "Database::Accessor::Param" GitHub\database-accessor-driver-dbi\lib/Database/Accessor/Driver/DBI.pm line 412.
So today I figure I better fix it. For once in the Perl world the problem and error message are quite clear. Seem I never did add in the 'Database::Accessor::Roles::Alias' into the ' Database::Accessor::Param' class so that is very easy to fix up.

package 
           Database::Accessor::Param;
        use Moose;
        extends 'Database::Accessor::Base';
++        with qw(Database::Accessor::Roles::Alias');
        has value => (
Just add it in an on the retest I bingo;
… ok 9 - 2 Fields and 2 parama retrieve SQL correct

ok 10 - 2 Fields and 2 parama retrieve params correct

So a real sort post-ette for today.

IMG_7113a.jpg

My First Talk at The Perl Conference in Glasgow 2018

I am giving a short talk on "How to become a CPAN contibutor?" at the The Perl Conference in Glasgow 2018. This is going to be my first ever talk at the European Perl Conference. I have already prepared the first draft of the slides for my talk. I will be doing final cleanup in the next few days. I have also got the hotel and train tickets booked.

For a change, this time I am taking my family with me. The plan is to have Glasgow City tour one day and one day Edinburgh City tour with the family.

Before I was mostly associated with London Perl Workshop for obvious reason as I live in London. Few months ago, I got the opportunity to attend German Perl Workshop 2018 in Gummersbach. I must say, I was very impressed with the way the event was handled by capable team of organisors, specially Jens and Roland.

Machine learning in Perl: Kyuubi goes to a (Model)Zoo during The Starry Night.

Hello all, this is a fourth blog post in the Machine learning in Perl series, focusing on the AI::MXNet, a Perl interface to Apache MXNet, a modern and powerful machine learning library.

If you're interested in refreshing your memory or just new to the series, please check previous entries over here: 1 2 3

If you're following ML research then you're probably well aware of two most popular libraries out there, Google's TensorFlow and a relative newcomer to the field but rapidly gaining widespread acceptance, Facebook's PyTorch.

The reason why PyTorch has gained so much ground on TensorFlow is in dynamic nature of that library. TensorFlow started as a static graph library (which is easier to optimize) and PyTorch went with dynamically allocated graphs and NumPy (read PDL) style of programming (with a robust GPU support and auto-differentiation of the gradients) that is as easy to debug as an ordinary Python's code.

Moose Never Stops Testing

Its new test day again in the Moose-Pen

Well starting with the 20_fields.t test case today and this is really just this so far;

my $in_hash = {
    da_compose_only           => 1,
    update_requires_condition => 0,
    delete_requires_condition => 0,
    view                      => { name => 'people' },
    elements => [ { name => 'first_name', }, { name => 'last_name', }, ],
};
my $container = {
    last_name  => 'Bloggings',
    first_name => 'Bill',
};
my $tests = [
    {
        caption => 'Fields',
        create  => {
            container => $container,
            sql =>
              "INSERT INTO people ( first_name, last_name ) VALUES( ?, ? )",
            params => [ 'Bill', 'Bloggings' ]
        },

        retrieve =>
          { sql => "SELECT people.first_name, people.last_name FROM people" },
        update => {
            container => { first_name => 'Robert' },
            sql       => "UPDATE people SET first_name = ?",
            params    => ['Robert']
        },
        delete => { sql => "DELETE FROM people" },
    },
    {
        caption  => '2 Fields and 2 parama',
        key      => 'elements',
        elements => [
            { value => 'User Name:' },
            {
             name => 'first_name',
            },
            { value => 'Address:' },
            { name => 'last_name', },

        ],
        create => {
            container => $container, 
            sql =>
              "INSERT INTO people ( first_name, last_name ) VALUES( ?, ? )",
            params => [ 'Bill', 'Bloggings' ]
        },

        retrieve => {
            sql =>
              "SELECT ?, people.first_name, ?, people.laste_name FROM people",
            params=>[ 'Bill', 'Bloggings']  
        },
        update => {
            container => { first_name => 'Robert' },
            sql       => "UPDATE people SET first_name = ?",
            params    => ['Robert']
        },
        delete => { sql => "DELETE FROM people" },
    },
];
$utils->sql_param_ok($in_hash,$tests);
Which translates into 13 test compared to the seven I had in the previous incarnation of the fields test case.

A litle Big Moose forward

It carry on testing developing day here in the Moose-Pen.

I left off yesterday with a new Test::Utils->sql_param_ok sub that is going to save a good deal of coding effort as I move forward once I get all the tests up to the new standard of curse, (one foot forward one and a half steps back)

Today I am going to carry on with alias testing this time I am checking basic field alias. To accomplish this I changed the 'elements' in my $in_hash to ones with field aliases;

$in_hash->{elements}= [
        {
            name => 'last_name',
            alias=>  'last'
        },
        {
            name => 'first_name',
            alias=> 'first'
        },
    ];
Next I want to change only the 'retrieve' test a little by update the sql like this

$tests->[0]->{retrieve}->{sql}="SELECT sys_users.last_name AS last, sys_users.first_name AS first FROM people sys_users";
to one that expects the SQL keyword 'AS' followed by the 'alias'. Then I just just the same test again and finally I change the caption of the tests so I can tell them apart and I run my test;

Better Moose Testing

Its go back and fix day here in the Moose-Pen

In a recent post I mentioned that I went out and read though a number of SQL syntax books to make sure I was doing things correctly in which SQL clauses (Where, Sort, Link) goes where. That go me to thinking was the system I was using to test the SQL on a real DB using standard SQL?

Normally I test with Oracle App Express as it work well on a Windows box and has a nice UI but I just noticed that I have some odd SQL in one of my test cases 15_alias.t;

I was starting with this hash;

my $in_hash = {
    da_compose_only=>1,
    update_requires_condition=>0,
    delete_requires_condition=>0,
    view     => { name => 'people',
                  alias=> 'sys_users' },
    elements => [
        {
            name => 'last_name',
        },
        {
            name => 'first_name',
        },
    ],  
};
And from my playing about I would generate this SQL
INSERT INTO people sys_users ( sys_users.first_name, sys_users.last_name ) VALUES( ?, ? )

Evozon Perl internship

Our new Perl internship started on the 9th of July, 7 daring new interns took on the challenge of becoming Perl developers. The internship will last for 4 weeks, during which they will get familiar with Perl and all that goes with it.

Perl is one of the pillars of Evozon, the company was founded in 2005 as a Perl and Java shop, expanding to other technologies over the years. After 13 years of Perl development we have plenty of experience and excitement for this language, something we want to share with this new group of interns.

We have one of the largest Perl teams in Europe and we’re proud to say that quite a few of our developers started their career with us, learning about Perl through internships just like this one. Now, they’re passing on the knowledge they’ve accumulated over the years to another round of Perl developers to be.

The internship will have two parts, a theoretical part where the interns will be introduced to the world of Perl by our trainers and a practical part where they will be working on an application built in Mojolicious with DBIx and MySQL, using the microcontroller Raspberry Pi. The microcontroller will have several types of sensors attached and will be able to monitor the temperature and humidity in our office.

Moose Back on Track Again

It back to coding here in the Moose-Pen

About a week ago I had this rather obtuse SQL expression;

(abs((people.salary + .05) * 1.5))*people.overtime+(abs(people.salary+.05) *2)*people.doubletime)

for testing parentheses. I figured I might as well re-use that little snippet of code else where as it is valid SQL anywhere you would use a 'field'.

One very odd place to use it would be in a sort if you want to order a list by how much over time and double time a person earned. So I added a new test to my '60_order_by.t' case;

...
    sql     => "SELECT people.first_name, people.last_name, people.user_id FROM people ORDER BY people.last_name, people.first_name",
},{
    key  =>'sorts',
    sorts => [$expression],
    caption => "Expression in Order by ",
       sql     => "SELECT people.first_name, people.last_name, people.user_id FROM people ORDER BY (abs((people.salary + ?) * ?))*people.overtime+(abs(people.salary+.?) *?)*people.doubletime)",
    params  => ['0.5','1.5','0.5','2']
}];
use Test::More  tests =>3;
my $utils =  Test::Utils->new();
$utils->sql_param_ok($in_hash,$tests);

Perl6 Readline update

A long time coming, this looks along /usr/lib64 and /lib/x86_64-linux-gnu for existing libreadline.so.* libraries, otherwise assumes v7. I'll add more directories as I find them, and make sure that it looks recursively in /lib if all else fails as part of the next release. I'll be adding comments to the GitHub issues shortly, as it should address most of the existing problems.

big Moose Done Tests (for now)

Still stuck in test mode here in the Moose-Pen.

Today I am re-writing the '47_dynamic_gathers.t' test. Looking at the code I cannot really use any of it as my Gather concept moved from two array-refs (gathers,filters) to just a singleton of 'Gather' made up of, elements and condtions.

The test is a little simpler now as all I have to test for is when I add a 'gather' the 'elements' and 'conditions' are correctly passed down from the DA to the DAD and when I add another 'gather' the current one is overridden by the new one;

To start I created two gathers;

my $gather = {        
       elements => [
            {
                name => 'first_name',
                view => 'People4'
            },
            {
                name => 'last_name',
                view => 'People5'
            },
            {
                name => 'user_id',
                view => 'People6'
            }
        ],
        conditions => [
            {
                left => {
                    name => 'last_name',
                    view => 'People7'
                },
                right             => { value => 'test' },
                operator          => '=',
                open_parentheses  => 1,
                close_parentheses => 0,
            },
            {
                condition => 'AND',
                left      => {
                    name => 'first_name',
                    view => 'People8'
                },
                right             => { value => 'test' },
                operator          => '=',
                open_parentheses  => 0,
                close_parentheses => 1
            }
        ]};
my $gather2 = {        
       elements => [
            {
                name => 'first_name',
                view => 'People4'
            },           
        ],
        conditions => [
            {
                left => {
                    name => 'last_name',
                    view => 'People7'
                },
                right             => { value => 'test' },
                operator          => '=',
               
            },
        ]};

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl and offering the modern features you’ve come to expect in blog platforms, the site is hosted by Dave Cross and Aaron Crane, with a design donated by Six Apart, Ltd.