blogs.perl.org

These are some answers to the Week 72 of the Perl Weekly Challenge organized by Mohammad S. Anwar.

Spoiler Alert: This weekly challenge deadline is due in a few hours. This blog post offers some solutions to this challenge, please don’t read on if you intend to complete the challenge on your own.

Since both tasks in this week challenge are quite simple, I decided to use only one-liners to solve each task, both in Raku and in Perl.

Task 1: Trailing Zeros

You are given a positive integer $N (<= 10).

Write a script to print number of trailing zeroes in $N!.

Example 1:

Input: $N = 10
Output: 2 as $N! = 3628800 has 2 trailing zeroes

Example 2

Input: $N = 7
Output: 1 as $N! = 5040 has 1 trailing zero

Example 3:

In my last blog post, there was a bit of a discussion in the comments about whether data in unit tests should be hardcoded or not.  Tom Metro scored the last point with this comment:

We always coach developers to use hard coded test data to the extent practical. When writing tests you have to unlearn a lot of DRY principles. We tolerate a lot more repetition, and factor it out sparingly.

and then I said that this really required a longer discussion than blog post comments would allow.  This, then, is that longer discussion.

So, first of all, let me agree whole-heartedly with Tom’s final statement there: “We tolerate a lot more repetition, and factor it out sparingly.” Absolutely good advice.  The only problem is that, as I noted previously on my Other Blog, we humans have a tendency to want to simplify things in general, and rules and “best practices” in particular.  I don’t think that Tom was saying “always hardcode everything in your unit tests!” ... but I think some people will read it that way nonetheless.  I would like to make the argument that this is a more nuanced topic, and hopefully present some reasons why you want to consider this question very carefully.¹

Cryptic Message

The communication system of an office is broken and message received are not completely reliable. To send message Hello, it ended up sending these following:

H x l 4 !
c e - l o
z e 6 l g
H W l v R
q 9 m # o

Similarly another day we received a message repeatedly like below:

P + 2 l ! a t o
1 e 8 0 R $ 4 u
5 - r ] + a > /
P x w l b 3 k \
2 e 3 5 R 8 y u
< ! r ^ ( ) k 0

Write a script to decrypt the above repeated message (one message repeated 6 times).

Even without reading the hint, the idea seems clear: for each column, the output should consist of its most frequent character. As usually, to count frequency, we’ll use a hash. To find the most frequent one, we’ll use max from List::Util.

Perl has some really great community websites. But a drawback to this is that the Perl community is centred around a few domain names, which means that it isn't as visible as some other languages. Most projects use github for development and CPAN for distribution, and outside those sites, they don't have much online presence.

One thing I think might help spread Perl around the web would be if different Perl projects had their own websites.

These are some answers to the Week 71 of the Perl Weekly Challenge organized by Mohammad S. Anwar.

Task 1: Peak Elements

You are given positive integer $N (>1).

Write a script to create an array of size $N with random unique elements between 1 and 50.

In the end it should print peak elements in the array, if found.

An array element is called peak if it is bigger than it’s neighbour.

Example 1:

Array: [ 18, 45, 38, 25, 10, 7, 21, 6, 28, 48 ]
Peak: [ 48, 45, 21 ]

Example 2:

Array: [ 47, 11, 32, 8, 1, 9, 39, 14, 36, 23 ]
Peak: [ 47, 32, 39, 36 ]

The specification somewhat lacks precision, but the examples are clear enough to clarify.

Peak Elements in Raku

The title is clickbait. I ran short of time this week and am ~~recycling~~^Wconsolidating comments, replies and thoughts. Let's talk about Books!

I would love a new PDL Book. One that's completely different from the original to maximize the surface of engagement to a new audience. As a "sequel", It would have the advantage of being able to refer the reader to the first book for longer explanations and be able to jump right into how to solve significant problems. brian d foy has just finished his Mojolicious book, so I bet he's got loads of free time on his hands. (although I remember him in the middle of writing it in 2018, so you may have to wait a bit)

I'm using the fabulous FFI::Platypus to interface to a C routine which uses caller-allocated buffers to return data. While FFI::Platypus transparently translates Perl arrays to C arrays and back, the buffers are used only to return data, so I only need the C-to-Perl conversion and not the Perl-to-C conversion.

The first step is to efficiently allocate a buffer of a given size in Perl (the last step, converting the retuned data in the buffer to Perl, is done straightforwardly with unpack).

If you do your due diligence, you'll find a link to an old PerlMonks post, which provides the following recipe:

my $str;
vec( $str, $length, 8 ) = 0;
$str = '';

I gave a talk at TPC 2019 based on my experiences speeding up the Perl test suite at room/roommate finding service SpareRoom, also serving as an introduction to the - just released at the time - Test2::Aggregate. The talk was a bit too dense, as I had prepared a pretty packed 20 minute presentation, only to realize a couple of days before (newbie speaker) that I had just 15 minutes real time excluding the Q&A. So, some attendees asked me to put up a blog post with the notes etc, and especially more about Test2::Aggregate, which is why I am writing this. I will try to give a longer and more detailed talk about the subject in one of the Perl conferences this summer.

In any case, the talk is up on youtube and gives an overview of the various lessons learned and techniques used while making our frustratingly slow 20 min test suite almost 10x faster, making a huge difference in our dev process:

The slide deck is available here.

Test2::Aggregate

These are some answers to the Week 70 of the Perl Weekly Challenge organized by Mohammad S. Anwar.

Task 1: Character Swapping

You are given a string $S of size $N.

You are also given swap count $C and offset $O such that $C >= 1, $O >= 1, $C <= $O and $C + $O <= $N.

Write a script to perform character swapping like below:

$S[ 1 % $N ] <=> $S[ (1 + $O) % $N ]
$S[ 2 % $N ] <=> $S[ (2 + $O) % $N ]
$S[ 3 % $N ] <=> $S[ (3 + $O) % $N ]
...
...
$S[ $C % $N ] <=> $S[ ($C + $O) % $N ]

Example 1:

The start of year 2020 didn't go well as planned. First my Dad was hospitalised and I had to make emergency travel plan to visit India. Luckily he is out of danger and back home. During this whole drama, the Perl Weekly Challenge got less of my attention. Thankfully I had loads of support messages throughout. Some offered to chip in so that I can focus on my Dad's health. I even missed my turn of editing Perl Weekly newsletter. It never happened ever since I joined the team of editors. Thanks to the chief edit, Gabor Szabo, I survived.

Another casualty of the January 2020, I missed submitting one Pull Request on everage in the month. I only submitted 22 Pull Requests. I have done this non-stop since October 2017. Sufferings didn't stop there, I couldn't get the monthly report published on the 1st Feb as per the tradition. It got delayed by 2 days.

Square Secret Code

The square secret code mechanism first removes any space from the original message. Then it lays down the message in a row of 8 columns. The coded message is then obtained by reading down the columns going left to right.

For example, the message is “The quick brown fox jumps over the lazy dog”. The code message would be as below:

tbjrd hruto eomhg qwpe unsl ifoa covz kxey

Let’s start with the test:

#!/usr/bin/perl
use warnings;
use strict;

use Test::More tests => 1;
is square_secret_code('The quick brown fox jumps over the lazy dog'),
    'tbjrd hruto eomhg qwpe unsl ifoa covz kxey',
    'encode';
        

LANraragi is a web application for archival and reading of manga/doujinshi. It's lightweight and Docker-ready for NAS/servers. There is even a standing offer from the author to send out a free sticker pack for the first person to run the linux/s390x docker image on a real IBM System 390.

Check out the release details and try out the demo

These are some answers to the Week 69 of the Perl Weekly Challenge organized by Mohammad S. Anwar.

Spoiler Alert: This weekly challenge deadline is due in a couple of days (July 19, 2020). This blog post offers some solutions to this challenge, please don’t read on if you intend to complete the challenge on your own.

Task 1: Strobogrammatic Numbers

A strobogrammatic number is a number that looks the same when looked at upside down.

You are given two positive numbers $A and $B such that 1 <= $A <= $B <= 10^15.

Write a script to print all strobogrammatic numbers between the given two numbers.

Example

Input: $A = 50, $B = 100
Output: 69, 88, 96

Several YAML modules allow loading and dumping objects. When loading untrusted data, this can be a security vulnerability, if this feature is enabled.

You can create any kind of object with YAML. The creation itself is not the critical part, but if the class has a DESTROY method, it will be called once the object is deleted. An example with File::Temp removing files can be found here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=862373

YAML::Syck had the option to disable this feature via $YAML::Syck::LoadBlessed for a long time. Since 2018, also YAML.pm and YAML::XS have this variable.

See also my blog post from 2018: Safely load untrusted YAML in Perl

In the past, this feature was enabled by default in all three modules.

This will now be disabled by default, to make sure that Perl's YAML libraries are, by default, more secure.

If you are using one of the modules to serialize/load objects, you have to set this variable now:

Only 100, please

You are given a string “123456789”. Write a script that would insert ”+” or ”-” in between digits so that when you evaluate, the result should be 100.

We can populate each place “between digits” with one of three possible values: a plus sign, minus sign, or nothing. To check all the possible permutations, we’ll use an indicator function similarly to The Knapsack Problem. In this case, though, there are three possible values, so we need to loop over numbers in the ternary numeral system.

The only operation we’ll need will be the increment, so we don’t need the full support for arithmetic in base 3. We can implement the increment ourselves: we start from the right of the number, change any 2 into 0 and move left. Once we find 0 or 1, we increment it and we’re done.

To create the expression, we just need to intersperse the digits with the operators. See the apply subroutine below.

Kind of like Moops but with less hacky parsing.

These are some answers to the Week 68 of the Perl Weekly Challenge organized by Mohammad S. Anwar.

Task 1: Zero Matrix

You are given a matrix of size M x N having only 0s and 1s.

Write a script to set the entire row and column to 0 if an element is 0.

Example 1:

Input: [1, 0, 1]
       [1, 1, 1]
       [1, 1, 1]

Output: [0, 0, 0]
        [1, 0, 1]
        [1, 0, 1]

Example 2:

We are really happy to announce that Curtis “Ovid” Poe will present a keynote at the 22nd Perl/Raku workshop in March in Erlangen!

Curtis runs Tau Station and is a long time contributor to the workshop.

The list of accepted talks has grown, with varied topics from “Progressing from Humans to Developers”, “A new Lisp, in Perl” and “Querying the Etherum Blockchain Nodes with Raku”. All accepted talks are listed here .

Since we still have some slots free for talks, we have extended the deadline for talk submission to the 3rd February 2020. If you have a topic you want to present, please submit your talk .

We’d also like to thank our sponsors for supporting the Perl and Raku community

Friedrich-Alexander-Universität - https://www.fau.de

United Domains - https://united-domains.de

Perl-Services.de - Renée Bäcker - https://perl-services.de

genua GmbH - https://genua.de

dpunkt.verlag - https://www.dpunkt.de

Well it looks like a wrap for PAWS XML as the last thing I am working on is getting the test suite to pass

ok 829 - Call S3->SelectObjectContent from t/09_requests/s3-select-object-content.request
not ok 830 - Got content eq from request
#   Failed test 'Got content eq from request'
#   at t/09_requests.t line 123.
#          got: '<SelectObjectContentRequest xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><InputSerialization><CompressionType>NONE</CompressionType></InputSerialization><OutputSerialization><CSV><FieldDelimiter>\,</FieldDelimiter><QuoteCharacter>\'</QuoteCharacter><QuoteEscapeCharacter>\"</QuoteEscapeCharacter><QuoteFields>ASNEEDED</QuoteFields><RecordDelimiter>\\n</RecordDelimiter></CSV></OutputSerialization><Expression>MyExpression</Expression><ExpressionType>SQL</ExpressionType></SelectObjectContentRequest>'
#     expected: '<InputSerialization><CompressionType>NONE</CompressionType></InputSerialization><OutputSerialization><CSV><FieldDelimiter>\,</FieldDelimiter><QuoteCharacter>\'</QuoteCharacter><QuoteEscapeCharacter>\"</QuoteEscapeCharacter><QuoteFields>ASNEEDED</QuoteFields><RecordDelimiter>\\n</RecordDelimiter></CSV></OutputSerialization>' 

I rechecked the API and my real world test and found the action 'SelectObjectContent
' it is a one off case in S3 that requires the root tag and the 'xmlns' to be present;

So a simple fix to the test got that.

When last we saw our heroes, what they thought was the brink of success turned out to be the precipice of hasty interpretation and now they are dangling for dear life on the branch of normalization! how's that for tortured metaphor!

If you use raw values for your k-means clustering, dimensions with large values or large ranges can swamp smaller dimensions and skew your clusters. The process of normalization tries to bring everything into the same range, usually [0,1], although your choices on how to transform the ranges are also significant. There is not always one best way to do it and, as usual, get familiar with your dataset and use your judgement.