Integrated Inconsistencies.

I will get it wrong. I will start off by saying that, not just because I am married and this sentiment has been conjugally programmed in me for years, but because doing things "my way" will not suit everybody. We approach life, programming, drawing from different perspectives, different analogies, and one method however disagreeable to one person, may be perfectly logical to another. Even our own actions and analysis show conflicts. Take a cup of tea. I drink from the top of the cup, but measure from the bottom. Take character position in programming code...we measure lines from the top, then character on that line. But when we write, we write one line at a time, populating columns in a line before going to the next line.

The same applies to drawing onto a screen. The reason I want to draw on a screen is because I want to be able to draw charts...following the example of Descartes, one plots a point on X,Y coordinates with the origin on the bottom left hand corner. x comes before y. The same screen is drawn from top to bottom, and the print coordinates are described as row, column. with the origin at top left. Then examine what happens if I were to draw an SVG graph according to the Perl Weekly Challenge 165 by Ryan Thompson. How many people realised that the SVG coordinate system the x,y coordinates are not the same as Cartesian coordinates? You did? Well done!...but I dare say plenty of people didn't.

I wrote Term::Graille, not because Term::Drawille was not good enough, but out of sheer indignation. You see, using Braille characters to draw "Pseudo pixel" graphics on the console would be simple but while I might desire a y-axis that increases as we go up as in charts, the dots in braille characters are confusingly numbered and bit 0 is the top of the character.

Drawing lines requires clever calculations. So I then took over Algorithm::Line::Bresenham, and found the original code appeared to take the y-coordinate before the x-coordinate...blasphemy, even though it made no difference to the actual results (it works both ways equally). You will (or perhaps will not) be pleased to know that x comes before y in the newer extended versions. I even inverted the textAt() character coordinate system in Graille; printing to canvas requires the coordinate system to have Cartesian origins, as text in charts often have to be placed next to the plot positions they target.

Graille Incorporates Turtle-like graphics and line drawing primitives, and in this case using Cartesian coordinates makes perfect sense. Mapping is simple and up and north are positive directions. It appears possible that keeping consistent axes should be acceptable for most needs.

Now that is fine if the data is sourced internally; often however, the data is not. Using external data relies on external paradigms, and one has to accommodate the logic of another representation. Images are read as raster from top down, y=0 is the top line. So plotting on Graille requires the image to be read in and then inverted to look right way up.

 for (my $x =0;$x<$canvas->{width};$x++){
  for (my $y=0;$y<$canvas->{height};$y++){
     $canvas->set($x,$y) # inverting image as the pixels are read
        if $image->GetPixel("x"=>$x,"y"=>$canvas->{height}-$y);

Then there is other bitmap data, font data or sprite graphics again represented top down, left to right. Conversion into blittable blocks requires this consideration. It is at this point I wonder if it is me that is going mad and whether I should stop trying to conflict with established paradigms. Then I realise that in a heterogeneous world conflict is inevitable, diversity is desirable, and confusion a natural and consistent consequence.



I continue updating Term:: Graille, extending drawille, integrating Turtle Graphics (as in the original asciimoo's version, but being in Perl, considerably more elegantly), Coloured lines, Image importing, Animations, 8x8 bitmap fonts, Heterogenous canvas etc, adopting a mainly Cartesian Coordinates for as much as possible. Maybe you think I am doing this completely wrong, but hey, I knew that all along. Version 0.07 coming soon!

The benefits of change, from an amateur's objective perspective.

As a bystander in the evolution of Object-Oriented Programming in Perl, and someone who is really only just starting to get the hang of Perl modules and packages (still not any good at it), I get really quite overwhelmed by ideologies. There is considerable debate about the right way to program things, the right style, the right direction that Perl should go. It is Vim vs Emacs, Atari ST vs Amiga, Mods vs Punks. Really one needs a language to do what one needs it to do, simply, quickly and consistently. For an amateur, Perl has been able to do exactly that for me. I code rubbishly, but hey, who's looking?

But when you look at the clever people in the Perl community, you wonder. You wonder whether you are missing out by not being engaged in the evolutionary process. It is bewildering. I confess I understand a mere fraction of the terms, the rationale, the higher order coding discussions. I have read about Corinna, marvelled at Moose's capabilities, gazed admiringly at Leonerd's prowess, but do not dip my coding toes into the water. You see, I cant swim.

But every body is like, "the water is warm and safe", and "c'mon, Saif, there are no trolls here!".

Ok, ok, let's start in the shallow end. I once had a plan for a collection of "PerlayStation Console Games" (you see what I did there?). Arcade Games for the terminal written in Perl. I could resurrect that while learning "proper" OOP...POOP! But I still have to figure out Slots, Roles, lvalue, (*gasps for air*).

Let's have a plan. Breathe. The way one might develop Perl modules, is first write code without modules that works, sort of like a feasibility test. Then, perhaps separate parts that can go into a module. That module is reused in other projects, and in doing so is tweaked to allow the extended capability. A successful module can be used portably across projects. Then the same program is developed using classes in other OOP paradigms, say Corinna, then perhaps Moo,etc.

Pong was developed, a game with Sprites. The sprites contained position and motion data and the info required to draw the sprite, all contained in a hashRef, which could be passed to subroutines for processing. e.g my $sprite={x=>10....};. This was a deliberate precursor for later conversion into a blessed object e.g. my $sprite=Sprite->new(x=>10,...);, all in all a seemingly painless transformation. The sprite modules that were created from that were initially classic Packages, then eventually as classes using Object::Pad. If I can figure it out I will also use Moo. For the moment the code versions remain monolithic until we have tested it/extended it enough.


Then we measure. Ease of use, readability, code length, performance and other parameters, trying to OBJECTIVELY determine whether to dive in and enjoy the waters, or just stick to shore where it is safe.

The times you want to avoid the short cuts. PWC-39

No spoilers here, just a rambling. There are probably countless times when one relies on the genius of others to complete tasks. If PerlWeeklyChallenges has shown me one thing, it is that there is probably nothing I can do that someone else can not do better, more efficiently, and more robustly. The key to good coding, one imagines, is to have code that survives not just the scenario presented in the challenge, but also accommodates other scenarios

The Hotel lights

Take the lights on puzzle. A first glance at the puzzle show that the times are all overlapping, so technically the lights should be on continuously from the first guest entry to last exit. So the simplest way to extract the time the lights were on is to get the earliest time and the latest time mentioned in the log. These can be extracted using four steps: load the log, extract the earliest and the last time; convert to minutes and subtract.

#!/usr/env perl
# Perl Challenge 39 Task 1
my $log =  # from the puzzle 
"1) Alex    IN: 09:10 OUT: 09:45
2) Arnold  IN: 09:15 OUT: 09:33
3) Bob     IN: 09:22 OUT: 09:55
4) Charlie IN: 09:25 OUT: 10:05
5) Steve   IN: 09:33 OUT: 10:01
6) Roger   IN: 09:44 OUT: 10:12
7) David   IN: 09:57 OUT: 10:23
8) Neil    IN: 10:01 OUT: 10:19
9) Chris   IN: 10:10 OUT: 11:00";

# extract earliest and latest times
my ($earliest,$latest)=(sort ($log=~/\d\d\:\d\d/gm))[0,-1];

#convertion function to produce minutes
sub time2minutes{
    my ($hours,$minutes)=split /:/,shift;
    return $hours*60+$minutes;
#print results 
print "Earliest time is $earliest,
Latest time is $latest.
Lights on duration is ", time2minutes($latest)- time2minutes($earliest);

Cheating is no doubt fun, gets the job done, but what if the data was non-contiguous, instead of what was presented in the challenge? We'd need to work out overlaps. The data was presented sequentially, conveniently for identifying overlaps, but what if it wasn't? Would it be better to to allow in your code to accommodate these possibilities and exceed the demands of the puzzle?

Extend the challenges

Tasks that may be completed using simple algorithms are great. Having entered the last couple of times it is clear that much can be done capitalising on previous skilful endeavours presented in CPAN. Doing a MetaCPAN search is trivial and solves many problems quickly. The RPN Challenge can be easily completed by using Math::RPN or Parse::RPN but doing so to solve a problem also fails in one component that appeals to me. Challenges exist to educate and push personal boundaries. Observing other folk's entries also entertain and inspire, so thanks you clever bods who also do the challenges. The greater challenge would be to include as many operations as possible with as little effort as possible.

RPN actually simplifies mathematics; for example, one could simply eval "$stack[0] $operation $stack[1]" and thus produce a generic translation for several built in binary operations, simply removing parameters from the stack and inserting results as needed. This way one could handle strings as well as numbers. For a comprehensive RPN calculator one would also need to include operations that manipulate the stack. And then there are other operations not included in the base Perl. Of course one could import Math::Trig, but seeing as we are building an RPN interpreter, why not code for functions like asin, tan etc in RPN? asin($var) can be calculated using atan2($var, sqrt(1 - $var * $var)). Covert that to RPN dup dup * 1 - sqrt swap atan2. Why would you do that? Of course it is slower, but the stack manipulation happens automatically, the whole thing can be done in one phrase, proves the capability of the interpreter, and gives you a warm fuzzy feeling inside.

Therein lies the problem. While challenges posed by someone maybe finite; the challenges you pose on yourself may have no end. Sleep beckons.

Circular observations

Selfies demonstrate one of the principal laws of physics, psychosocial development of teenagers and programming. The act of observation alters that which is being observed. It becomes more complicated if the observer is observing itself. As the observer changes, so might future observations particularly if the tool being used is known to impact the parameter that is being measured; rapidly one loses hope that any meaningful data will be obtained from such a direction of development. Sounds gibberish? Look at the collection of "duckfaces" on your teenage daughter's phone (if she ever lets you near it) and I am sure you will find no resemblance of those images to anyone you would recognise as your offspring.

Let's do it any way

It is so with programming. But let's not let that stop us from still trying eh? The problem I alluded to in my last blog post regarding memory leaks in some GUI back ends offers such an opportunity. I had observed these leaks when trying to use timers to do animations. The way these animations were created was by creating a simple svg file, and altering that file in Perl, saving it, and inserting that file into an image panel, and repeating. A slowdown was noted with some back-ends , and using top, one can see the memory being consumed by that Perl interpreter process progressively rising.

I want to automate this measurement, and visualise the output graphically. But I don't know the pid of the process until I start the application. But my application knows its own pid. And my application itself can generate graphics. It can measure how much memory it uses, storing the data captured in an expanding data structure (an array) using changing amounts of memory, and using a subroutine that takes time and memory itself. It can then generate an SVG chart which gets more data points added each cycle, saving it and reloading it using processes that itself are suspect in memory leaks, and finally rendering this graph on the leaky UI. A fun-filled fool's errand that may deliberately exaggerate leaks thus demonstrating and identifying problematic systems while not, at first glance, offering fine grained analysis.

The pmap <options> <pid> system command (Linux).

$$ in Perl gives us the Process ID. Passing this as a parameter to pmap delivers a running application's memory footprint. It not only describes the total memory it uses but also subdivides the amount by internal structures which may give more clues as to what may be hungry for RAM. We can (for now) discard all the extraneous data and focus on the last line of the output (containing the total memory used) and progressively generate a chart of the excess memory being used with each cycle within the application and deliver it to the image panel widget in the UI.

 sub memRequired{
     my $mr=`pmap $$`;             # call pmap
     $mr=(split('\n',$mr))[-1];    # get last line
     $mr=~s/[^\d]//g;              # just keep numeric portion
     return 1*$mr;

Keeping an eye on the ball

qt Chart.pngThe animation script from before is now modified to allow selection of which animation is being done. The animation may be stopped without stopping the timer, and It is also possible to overlay the chart on top of the other SVG animations already playing. So I have done a couple of simple tests. It is clear it is not the timer objects themselves that are directly involved in the memory leak. It is the Image panel update process. Tk and Wx need ImageMagick...but Tk leaks more than Wx. In these tests Qt appears the leakiest (Figure 1) although not obviously using ImageMagick.

GT3 Chart.pngGTK3 and GTK2 both lose no extra memory with SVG animations generally. The slideshow (jpg images once updated every second) appears to show transient extra memory used, going up in step wise fashion, as if dependent on the size of the image displayed (Figure 2).

Tk Chart.pngFor the backends that do use memory while playing SVG images there appears to be a consistent amounts of memory lost each time. The images have varying complexities, but this seems not to matter...all the svg images are 200px vs 200px. Most animations in these tests update 10 times per second, but clock and the slide show updates the frame once a second. Switching between the 1/10 sec and 1 sec produces a predictable patterns of memory leak.

I would like to see if there are other ways to put an image into a frame other than using Image::Magick. ImageMAgick does support a huge number of formats though. I would prefer to be able to migrate to Imager. This seems more tightly integratable into Perl.

Re-Entry of GUIDeFATE into CPAN

Any way the timer handling is now more or less uniformly supported across Gtk2, Gtk3, Qt, Wx and Tk. I would not use it for animations in anything other than using the Gtk2/3 options. GUIDeFATE for almost the last year has been not synced with CPAN. Version 0.13 will see it back on GitHub but with some changes. I will be removing the sub-modules from the base path into a folder of its own, so GFTk becomes GUIDeFATE::GFTk, starting 1st October. The latest version on GitHub will also be updated roughly at the same time.

Time Flies, Memories leak...GUIDeFATE getting a timer

It appears that I have not had single blog post for 1 whole year. Let's face it, there isn't enough time in the day to everything one might want to do. Procrastination steals more than just time though. The longer one is away from a particular activity the more difficult it is to return to that activity physically or intellectually. Some of you will recall that I developed what I claimed to be the world's simplest GUI designer, GUIDeFATE. This was the topic of a couple of presentations already but as all such projects, remains in a state of flux.

One key component of a GUI is a continuous loop that keeps monitoring human interactions and responding to this by updating the interface. What if we wanted the interface to update continuously, like a monitor, without a connection with a human input trigger? It is certainly not possible, for example to create an animation by just doing foreach(1..$x){sleep $interval; updateImage();}. This merely blocks the GUI mainloop, until the loop is completed with the display updating only after the loop completes and the control goes back to the GUI's mainloop. Instead one needs an event monitor, and more specifically a timer. The ideal scenario may have been to use something like AnyEvent or Ev, but sadly this can not be applied to all the backends; at least I do not know how one might do this. What follows is my usual trawl through the internet to source examples of timers in GUI applications with each of the various backends.

Plug-in Timer

I discovered that Tk and Gtk can work with AnyEvent/AE fairly easily. Qt and Wx have their own way to create timers. While Qt seems to be supported in the documentation for Any::Event, the general recommendation appears to be not to use Qt. Qt Slots have to be predefined and can not be made on the fly. A little jiggery pokery and we have a simple way to abstract timers through multiple back ends. BUT all is not as straightforward as one would like, as I discovered when trying test animations in various backends.

To abstract the timer in the various supported back-ends, Gtk3, Gtk2 (thanks chanio), Qt4, Wx from the UI string requires a single line added to the textual representation of the UI e.g.

my $window=<<END;
|T Animation               |
| +I--------------------+  |
| | clock.svg           |  |
| |                     |  |
| |                     |  |
| |                     |  |
| |                     |  |
| |                     |  |
| +---------------------+  |
|                          |

timer 100 ecg 1


GUIDeFATE detects the timer line, and in the case above, executes the subroutine "ecg" every 100 milliseconds. Any::Event (gtk or Tk) or Wx will happily accept a callback function reference from a string...e.g. \&{"main::$functionName"}, Qt needs a predefined callback subroutine as a slot. While this subroutine could then be used to jump to the desired subroutine, there is some inflexibility in that the number of timers that one could create are limited.

Testing timers.

I figured the best visual way to test a continuous timer would be to make a slide show updating an image window; or perhaps using a image that is updated to make an animation. The easily portable way to make an animation in Perl (without added dependencies) I figured, would be to make SVG images with only minor textual changes (something Perl is good at) required to make a graphical change. Here is GUIDeFATE being used in an application running 4 different animations made by continuously redrawing an SVG file and loading the recreated image into a frame.


When I tested the timers for such animations I found 1) that Wx, Tk and Qt applications progressively used bigger and bigger amounts of memory, with Wx particularly slowing down to a stand-still after a few hundred timer loops 2) Gtk and Gtk2 seemed to use much the same amount of memory regardless of how long it ran for. Now Tk and Wx both used Image magick to put the images into the image panel. Could this be the cause? more testing is necessary.

Things to do

GUIDeFATE 0.13 is not yet on Github or CPAN...I need to sort a few things out first.

1) One of my targets is to restructure GUIDeFATE distribution so that it does not pollute the namespace as much as it does now. I would like to move the back end modules so that they are not in the root names space e.g. GUIDeFATE::GFWx. Before that (I think) I need to wipe the old GUIDeFATE from CPAN...not sure how to to do that without creating havoc.

2) I need a few methods to delete timers, change intervals, or the callback functions...should be easy, but need to to choose methods that will be easy, and of course test them.

3) I need to test on non-Linux machines...I have no Windows PCs or Macs.

Hopefully will have this ready by the next Perl Meeting, who knows?