Finance::QuoteDB 0.15 released

Please note that I have released Finance::QuoteDB 0.15 to CPAN.

Finance::QuoteDB is meant as a fullblown database application for
maintaining stock data. It allows anyone to easily create and update a
stock database. The information is gathered by using Finance::Quote
and the database is created and maintained by use of DBIx::Class.

The config-file for GeniusTrader use is generated automatically.
Interface to R is planned to be integrated.

Cascading config parsing

Prompted by some comments about how to handle RAW files from cameras, I revisited App::imagestream , the program I use to automatically publish all images I touch to a gallery page on my website.

An experiment that I run with this program (and with App::fritzgrowl as well) is to specify the configuration information via POD. There is code in Config::Spec::FromPod and Config::Cascade that takes POD and turns it into a hash of hashes containing the configuration model. Then there is more code in App::ImageStream::Config to turn this model into a DSL for a config file, a parser for Getopt::Long or simply the defaults. Config::Cascade then fills out the values, starting with the most specific values coming from the command line, the less specific values coming from the config file and the least specific values coming from the application defaults, all driven by the documentation.

Perl memory management...

Does Perl run out of memory?

Today I got an email from someone saying “I was told by a person who used Perl for computational genomics applications that it was running out of memory, so he switched to C++. What’s your thoughts on running out of memory in Perl?”

Just for posterity here is my reply (please note I’m no expert on this sort of thing and have never had the problem) was…

use types

Now that I have "almost" fixed the remaining compiler bugs, I wanted to use the existing framework to enable the possible speedup by using types, esp. low-level internal types. C-style integers and double scalars are used internally in B::CC instead of full Perl IV's / NV's if declared as such, and thus greatly improves execution speed. Esp. for inlinable arithmetic and comparison blocks.

Only at the end of such a block (and only when really needed) the calculated C vars are written back to the perl pad. So I needed better Opcodes flags to define which ops read or write pads, and more possible optimization hints. Most of these flags should be added to core later, when the time will come to speed up not only the B::CC compiler, but also the internal perl compiler.

But how to declare types?

qr/STRING/msixpodual or qr/STRING/mixuploads ?

I released perl 5.13.10 today. I might write more about that later. But one significant change in it is that Perl now has many more regex flags.

So I wrote a short one-off script to find out what words I cound construct from the flags.

Now it just gives you one word that contains as many of the flags as possible, and gives you the remainder. What would be more interesting would be to detect cases where multiple valid words can be made from the flags. E.g. "mix" and "uploads". It just detected that by accident.

I leave that as an exercise for the reader.

Plack::Middleware::NoMultipleSlashes

Ignited by comments on IRC and in the Israeli Perl Mongers mailing list, I've sat (for 10 minutes), wrote and published a new Plack middleware: Plack::Middleware::NoMultipleSlashes. Let me explain what it is.

Apparently you cannot count (by RFC, even) that the paths http://mysite/ and http://mysite// will lead to the same place. This is very tricky, because it means that the framework you're using (Dancer, right? :) shouldn't clean those multiple slashes for you.

So, I figured "why not let Plack take care of it?". I wrote Plack::Middleware::NoMultipleSlashes (a single line of code, really), that cleans those multiple slashes for you, wherever they are in the path.

To use it with Dancer, just add the following to your config.yml file:

plack_middlewares:
- [ NoMultipleSlashes ]

Done!

Thank you, Plack! :)

BTW, if you still haven't gotten around to reading Franck Cuny's latest post, it's right here (PSGIChrome).

Win32::Wlan released

Win32::Wlan has now escaped onto CPAN. I've redone the whole structure. There now is Win32::Wlan::API, which is a very thin layer over the Microsoft Wireless API. Win32::Wlan itself provides an object that handles the initialization and deinitialization of the API and provides convenient access to the first (and highly likely, only) wireless connection of the computer.

Now I have to write a convenient "snapshot" program to invoke to "mark" a place, and another program to recognize those wireless snapshots and to associate them with a location. When a location is then recognized, this could trigger scripts that configure other programs for the proper proxy settings.

Using external installers with CPAN Clients

A lot of work is currently underway to improve CPAN. One longstanding issue is the handing of external dependencies (the "libfoo" problem.) As I understand it, although CPAN distributions may test for external dependencies, existing build tools are unable to install non-perl libraries, header files or compilers.

Resources to solve these problems are available, but in places not usually considered by CPAN toolmakers. I'm referring to native OS package repositories. While natively packaged CPAN distributions necessarily lag current CPAN offerings, they meet non-perl as well as perl dependencies, and are generally troublefree to install.

To give the example I know best, Debian's perl repository covers roughly 10% of CPAN, including much of CPAN's best. This enormous investment in labor, toolchain development and community building results in high-quality packages that even new users can install easily.

As a developer of a perl audio application available for Debian, I've encountered problems managing dependencies when administering systems that mix distro and CPAN sources, even when using local::lib.

Executing code at the end of runtime.

The Problem

There is a lot of call for the ability to execute code at the end of runtime. An example of this is Moose when you call __PACKAGE__->make_immutable(). Wouldn’t it be nice if this were just automatic? Other places this can be useful are in testing frameworks such as Test::Class or Fennec. These testing frameworks use a ‘runner’ to run test structures defined in your tests.

The obvious (but bad) solution

Depending on your use-case for code that runs at the end of runtime, there are a couple possibilities. One possibility is to use an END block. Most people should know by now that END {} blocks are bad, simply read the perlmod section. There are just too many edge cases to account for, not to mention conflicting END blocks.

How Test::Class does it

What's New in WebGUI 8.0 #4 -- CHI Cache

Caching is a tricky business. Having just one kind of cache won't work, because the production environment will greatly determine the most efficient caching system. A distributed production environment would be best-served with a distributed cache. A smaller, single-server environment could use a simple shared memory cache.

Enter Jonathan Swartz's CHI module, the greatest Perl module to provide a unified caching interface. CHI is the DBI of caching: It presents an API, and delegates to CHI::Driver modules to perform the heavy lifting. It provides a layered caching system, allowing you to have a faster, more volatile cache in front of a slower, more persistent cache. It also provides a variable expiration time, preventing a "miss stampede" where all processes try to recompute an expired cache item at the same time.

By integrating CHI cache into WebGUI, we have the ability to provide any caching strategy that CHI can provide. We get Memcached, FastMmap, and DBI drivers (and more drivers can be written).

Making Perl location-aware, one bit at a time

One of my long-term ideas is to make my laptop more location aware and to have something like cron except for locations instead of time. It will be useful to automatically reconfigure my network and proxy settings in Firefox and Thunderbird and to prevent information leakage. Also, my laptop can automatically synchronize its minicpan and git repositories with the mothership when it detects that a "fast" or "local" network connection is available.

As I'm not really interested (yet) in the physical location, it is enough for the laptop to know what wireless networks are visible to determine how it should behave. Later (or once GPS sensors become commonplace in laptops, or Perl becomes commonplace on GPS-enabled devices), adding GPS detection etc. will also become interesting.

HTML::Formatter release

I've pushed out an initial release (version 2.05) of HTML::Formatter which covers the incompatibility with current perls and RTF formatter initialisation.

I'll work through the other issues as I get the opportunity - but the incompatibility with current perl versions was the reason I took on maintenance, so I wanted to get a release with that fixed out quickly.

ActiveState PPM index + download stats

ActiveState have updated their PPM index page:

http://code.activestate.com/ppm/

As reported http://www.activestate.com/blog/2011/02/ppm-index-new-way-browse-perl-packages.

It is interesting to see which are the popular downloads and as an author the number of downloads of your own modules. They also have a nice chart of which OS's the module has been built for.

Not so sure that the example in the article - DBD::Mysql Failing on OSX - is such a good showcase! (looking at the report just seemed that mysql_config wasn't in the build servers path or something, so a setup issue, not a module issue).

But that aside it's interesting to see how far PPM seems to have come.

Syntax police?

In a small script someone wrote at work I saw the indirect pattern of new Object, instead of the more correct form of Object->new. When I inquired (okay, I said "WTF?!"), he said that he just copied the synopsis of a module. Oh, right.. some of the synopses (plural of "synopsis", bet you didn't know that!) still have some outdated syntax examples.

I'm not gonna write about how we should all update our PODs to remove syntax that hasn't been (or shouldn't have been, at least) written for the last 10 years, even though I should! I wrote to talk about the reply my co-worker got when he opened a ticket asking for the synopsis to be updated, per my suggestion.

He got the reply "what is this, the syntax police?" Perhaps half-jokingly, but still problematic, IMHO.

Police? No. Neighbors, family members and friends? Yes!

is_almost()

I struggled with a problem where a given method would return an array of array refs of data, but the order (and sometimes presence) of array ref elements were sometimes slightly different. This is because this code needed to test real data and I could not mock the results. After giving this some thought, I realized I wanted something like the Levenshtein edit distance for data structures. Marcel Grünauer suggested that each element get assgined a unicode character. This solves my problem nicely with the following code ...

Non-functional perl code testing - automated code review

I saw a very ugly Perl code, and I think that is possible because no code review exist in project. But always we have a very little time for human code review. I search CPAN modules for basic level code review by machine. I hope that these modules help Perl people to avoid writing ugly code and help team leaders minimize time to code review. I try to collect founded modules in github repository and now want to describe what we know if all these test passed:

Portable plugin apps?

I might be working on a project for a friend shortly. They have lots of experience with developers who use Drupal - and I get the feeling (could be wrong) that you can semi-plug-and-play with Drupal?

So..
Want a wiki - use X plugin
Want a user system - use Y plugin
Want user profiles - use Z plugin
Want user gallery - use A plugin
Want a blog - use B plugin

So I started looking around, and there are plugins for specific frameworks, or standalone applications written in a framework, WebGUI CMS seems to have a lot of features but I want to use my framework of choice (and experience!).

Having been playing with Plack it got me thinking...

Just as Plack/PSGI sits between the webserver and your code, could someone (brighter than me!) come up with a standard for sitting between a framework and an app (blog/gallery/wiki/forum etc)?

CSS selector goodness in Mojo::DOM

Now that we've seen how easy Mojo::DOM makes parsing html, let's take a closer look at the css selector goodness it provides.

Here's a fairly verbose html sample for us to work with:

First, we initialize and parse the file:
use File::Slurp 'slurp';
use Mojo::DOM;
my $dom = Mojo::DOM->new->parse(scalar slurp 'some.html');

Getting all the articles' contents, of course, is easy:
$dom->find('li a');

But we can do better than that. Let's say we want only the article titles that have page anchors:
$dom->find('a[href^=#]');

Nah, let's get the article titles that link to external urls:
$dom->find('a[href^=http]');

How about only article titles that link to .net domains?
$dom->find('a[href*=.net/]');

We can also get the page anchors themselves:
$dom->find('div.article a[name]');

It could be that some articles have no text content; let's single those out:
$dom->find('div.article p:empty');

An amazing Javascript graphics package: Protovis

Hi Folks

I /still/ hate having to sign in twice in order to post. Grrrr.

Anyway, see this: http://vis.stanford.edu/protovis/

Examples: http://vis.stanford.edu/protovis/ex/

Slide mouse horizontally over this chart: http://vis.stanford.edu/protovis/ex/index-chart.html

However, let's not develop oodles on incomplete Perl interfaces, p-l-e-a-s-e. After all, we don't want another SVG debacle, do we?

Duplicating blog posts between blogs.perl.org and own blog?

I have my own blog, on which I post about Perl-related stuff along with other things. I also have a blog here on blogs.perl.org (obviously, as I'm posting on it now).

When I post something Perl-related, I'd like to post to both, as blogs.perl.org makes it more likely to be seen by the Perl community, many of whom probably don't follow my own blog, but I don't like the idea of manually posting it to both really. It's also a bit of a pain for any comments to be split between the two.

Solutions that come to mind:

  • Automatically copying posts in the Perl category on my own blog to this blog somehow; or a script that posts a summary along with a link to the original post on my blog
  • The above but in reverse; post here and have a summary & link posted to my blog
  • Not use this blog, and attempt to get the feed for the Perl category on my own blog listed on planet.perl.org, and hope that helps make those posts visible to the Perl community

I can't be the only one who has their own blog but also wants their content on blogs.perl.org for better visibility within the Perl community - anyone have any suggestions?

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl with a graphic design donated by Six Apart, Ltd.