Perl in a Trading Environment: A Multi-Pronged Success Story

Sean Blanton will give a talk at YAPC::NA 2012 described as:

Perl has proven itself in our start-up trading strategy in many different areas: system and application administration, networking, data collection, reporting, website and recently analytics with PDL.

For every task, we have evaluated the best tool for the job and decided on Perl because of the wide variety of available **tested** modules, and the ability to share code for very different tasks. Perl’s close relationship with C is a big plus. A set of generic services built up using Moose and Bread::Board has been particularly effective for code reuse. All of this helped reduce implementation time and free resources for other tasks, such as strategic analysis and planning, which in turn helped accelerate revenue growth.

[From the YAPC::NA Blog.]

If you look on every exit as being an entrance somewhere else ...

I didn’t write anything specifically about Perl this week, but I did broach a topic in my other blog that you may find interesting if you’re a professional programmer.  Check it out, if you like, or not.  If nothing else, it has a reference to a great blog post which is much better than mine. ;->

The Triumphant Return of Zoidberg -- A Modular Perl Shell

After more than a year of hoping to be able to say this, here goes: Zoidberg is back!

What is Zoidberg you ask? Well its a Perl shell, of course. Think of the fun of it all:

mv($_ => lc($_)) for grep /[A-Z]/, <*>

If you use a reserved Perl word first or if you wrap code in a block it is interpreted as Perl, if not then its a shell command. Along with having most of the things you would expect from a shell, it features plenty of extra bells and whistles, like a multiline input system (when Term::ReadLine::Zoid, part of Bundle::Zoidberg, is installed) and automatic splitting of variables to arrays like @PATH which is just $ENV{PATH} split on :.

When used as your login shell, it can be used from inside other programs like vim.

:!cat % | {/^\S/}g | wc -

Further, it is easy to extend with the Zoidberg::Fish plugin system.

use B::Stats to check for less bloat

I'm constantly concerned about small modules or p5p adding more and more bloat, especially "innocent" dependencies. The authors obviously do not care about memory on small devices or vmware hosting with 256MB, and since Moose is so popular obviously also do not care about serious run-time penalties loading all this cruft for almost no gain.

My top concerns are:

1) use warnings

which depends on warnings::register and Carp, which loads a whole bunch of big generally unnecessary warnings category hashes. And XSLoader. And B if compiled. With B::C I introduced -fno-warnings for 5.13.5 to save 68KB executable size on 32-bit. Uncompiled the numbers are of course obscenely higher.

2) use Carp

Best of two worlds - Drupal and Modern Perl happily married to build content driven web portal

Maxim Gregoriev will give a talk at YAPC::NA 2012 described as:

There was a goal to deliver a web portal where US Department of Energy WAN users can easily find all network performance metrics and where they can share or get some advice from the networking “wizards”. The WAN monitoring bits of data are gathered from the mesh of decentralised multi-domain network monitoring web services called perfSONAR-PS. The project we will be talking about is called “E-Center” and it  provides a higher level aggregation of the monitoring data with anomalous events detection, forecasting and fluid data visualization.

[From the YAPC::NA Blog.]

The Ten Minute Thinking Rule

What’s the problem?

This idea was shared with me during a retrospective session at $employer when I was expressing some frustration at people starting to:

Ask first, think never

I’m at least as busy as the next person and I have my own mountain of work to complete. Despite being regularly cantankerous, grumpy, cranky and disagreeable people still come to me with a variety of questions … I think I have a good combination of detective skills, google-fu and luck that makes me appear more magical than I really am.

The Ten Minute Thinking Rule

Simply put:

You may only interrupt another co-worker to ask a question if you have sat on your own for ten minutes and tried to figure out the solution yourself

Corollary

When asked, you must be able to demonstrate that you’ve made some effort

The Ten Minute Thinking Rule isn’t an exercise in pretending you’re thinking.

Some ‘thinking’

Thinking also includes basic investigations. Consider some of the following:

Perl Calltrace

How to get the abs path of the script that do use modulename ?

  • In the module:
 1 package foo;
 2 use common::sense;
 3 use Cwd 'abs_path';
 4 
 5 sub program {
 6     say abs_path($0);
 7 }
 8 
 9 1;
 

Activate ipv6 on Windows

$ cpan IO::Socket::INET6

t/blocking.t .......... IO::Socket::INET6: socket: Address family not
supported by protocol at t/blocking.t line 34.

t/blocking.t (Wstat: 27136 Tests: 0 Failed: 0)
Non-zero exit status: 106
Parse errors: Bad plan. You planned 2 tests but ran 0.
t/host6.t (Wstat: 27136 Tests: 0 Failed: 0)
Non-zero exit status: 106
Parse errors: No plan found in TAP output
t/io_sock6.t (Wstat: 27136 Tests: 0 Failed: 0)
Non-zero exit status: 106
Parse errors: Bad plan. You planned 20 tests but ran 0.
Files=9, Tests=10, 14 wallclock secs ( 0.09 usr 0.05 sys + 1.72 cusr 2.23
csys = 4.09 CPU)
Result: FAIL

$ netsh interface ipv6 install
The following helper DLL cannot be loaded: DGNET.DLL.
Ok.

$ cpan IO::Socket::INET6

All tests successful.
Result: PASS
SHLOMIF/IO-Socket-INET6-2.69.tar.gz
/usr/bin/make install -- OK

Stackato - Bringing the Cloud Back Home

Ingy döt Net will give a talk at YAPC::NA 2012 described as:

Stackato is a private/personal PaaS solution from ActiveState. Get all the simplicity of deploying your favorite Perl (and many others) apps, like you get from DotCloud or Heroku. The difference is you can host it anywhere you want, from your laptop to Amazon EC2. It’s your software, and its free.

In this talk Ingy döt Net will get everyone in the audience up and running their favorite Perl web apps on their own private PaaS in minutes. Fun!

[From the YAPC::NA Blog.]

CPAN Testers Summary - November 2011 - Sounds of the Universe

November turned into a rather more traumatic month than I would have liked. It didn't start well with the old hosting company issuing us a invoice for 1 year's hosting, which we had previously cancelled due to their appalling service regards our loss of data. They now claim we still owe them a month, regardless of the month outage we suffered due to their incompetence and failure to replace our mirrored HDD before the second one failed. I'm now seeking compensation, but I'm not hopefully it'll get very far.

The SQLite issues are still surfacing. Following a complete rebuild of the SQLite database, Andreas is still seeing errors for some searches. Though not all, so those of you who use the SQLite download, may see intermittent faults. I have no idea why this happens and if there are any SQLite experts out there, feel free to get in touch.

How to parse HTML, part 2

This is the second of a series of posts that details a Marpa-based, "Ruby Slippers" approach to parsing liberal and defective HTML. This post assumes you have read the first post.

First, reduce the HTML to a token stream

Most computer languages can be viewed as a token stream. HTML is not an exception. HTML tokens can be blocks of text; comments and various other SGML entities; HTML element start tags; and HTML element end tags. The HTML token stream is unusual in that some of its tokens can be quite complex internally.

In parsing computer languages, it is a frequent practice to divide the work between a tokenizer ("lexer") and a high-level parser . The lexer takes the raw input and turns it into a token stream. Tokenizing HTML is a difficult job, and one for which there is an excellent CPAN module: HTML::Parser . Marpa::HTML relies on HTML::Parser to do its tokenization.

Marpa::HTML

CPAN Testers - "Can't locate FindBin/libs.pm in @INC"

Following my recent update to Catalyst::Plugin::ErrorCatcher I’ve taken some time to scan through the CPAN Testers Reports for the module.

I have to admit to being somewhat perplexed at one of the common test failures [example report]:

Can't locate FindBin/libs.pm in @INC

I’ve specified the library as a ‘TestRequires’ dependency in my Dist::Zilla dist.ini file:

[Prereqs / TestRequires]
DateTime = 0
File::Slurp = 0
FindBin::libs = 0
...

The generated Makefile.PL contains this as a ’BUILD_REQUIRES’ dependency:

Managing Chaos with Perl

Ben Mouw will give a talk at YAPC::NA 2012 described as:

Booking.com is #1 online travel agency in the world (in terms of the number of online hotel room nights sold)  

In this talk I will discuss how we manage to run a large scale website using almost exclusively Perl.

I will cover our setup for running the business with Perl and Open Source, what we have done to remain agile while growing our team to over a hundred Perl developers, and some of the challenges we have faced in our rapid growth

[From the YAPC::NA Blog.]

Using system or exec safely on Windows

Passing a list of arguments to another program on Windows in perl is much more complicated than it should be. There are several different issues that combine that lead to this.

(mostly copied from a post I made on PerlMonks)

First is that argument lists are always passed as a single string in Windows, as opposed to arrays like on other systems. This is less of a problem than it appears, because 95% of programs use the same rules for parsing that string into an array. Roughly speaking, the rules are that arguments can be quoted with double quotes, and backslashes can escape any character.

The second issue is that cmd.exe uses different quoting rules than the normal parsing routine. It uses a caret as the escape character instead of backslash.

Yet Another Tel Aviv Perl Mongers Meeting Report

I have virtually (and somewhat physically) disappeared for a few good months, whether it was from the development community or from friends. I've needed a break from stuff, from planet Earth.

One of the things I had the privilege of doing upon returning to the surface is to organize our next Tel Aviv Perl Mongers (TA.pm) meeting. I was fortunate enough to have the help of Shlomi Fish and Gabor Szabo. I also had the pleasure of having Erez Schatz and Ido Kanner stepping up and offering to do a few talks!

2011-11-30 22.07.21-15.jpg

Ido's talk is available online.

You can read Shlomi's report and Erez also mentions in his last post.

A Zoidberg Story

I just read Buddy Burden’s recent post A Random Story and it was fun to see that his reasons for adopting Data::Random was the same as my reason for adopting Zoidberg: failing tests. The difference was that his failing tests were his own and mine were in Zoidberg.

I was a few years into using Linux and I had really fallen in love with Perl, which was my first real programming language (after Maple, Mathematica, and LaTeX), but I had never really gotten the hang of Bash. Since Perl was used in scripting, and even things that looked like shell scripting, I wondered if there was a Perl shell. I looked and I saw a few Zoidberg and Psh stood out as being functional.

Now I can’t remember what it was, but Zoidberg was the one that kept my attention, there was probably something, but I can’t remember anymore. It had a problem though, a failing test on installation through CPAN. I could install it through apt though, so no problem, right?

Test Automation Workshop

Gabor Szabo will run a Test Automation Training Workshop at YAPC::NA 2012. The workshop will be for the two days prior to the conference (Monday and Tuesday June 11 and 12).

AI Challenge 2011 - Python, Ruby, PHP, Perl

The AI Challenge Fall 2011 (sponsored by Google) is all about creating a computer program (in any language) that controls a colony of ants which fight against other colonies for domination.

Is there in truth no Perl?

While working on my Tel Aviv.pm talk about DBIx::Class talk (more to follow on that), I met an acquaintance of mine from a deceased start-up I worked on a few years back. We chatted a bit, he said he's still writing in C/C++ (mainly GUI stuff), and when I mentioned I work with Perl nowadays, he said "Oh, I love Perl, I wish there were other languages like it". I, as usual, remarked that there are the Pythons, and even PHPs of the world, and he surprised me by saying "no, I mean really like Perl".

Announcing MooseX::Types::NumUnit

I use Moose to write scientific simulations, including one very large simulation with a user api. To this point all of the numerical quantities, kept in attributes, needed to be of Num type. This always meant an implied covenant between the me and the users, which was to use SI units.

However I have a few quantities that I want to use eV units, which makes a lot more sense. Therefore I setup a simple type with coercion to accept a string num eV and coerce it to a number given by qe * num. This got me to thinking, why can’t I do this for all my units?

To answer this need, I present MooseX::Types::NumUnit which provides a couple static types, but also provides the function num_of_unit, which creates anonymous types which will automatically coerce a string to a number of the desired unit. For example:

About blogs.perl.org

blogs.perl.org is a common blogging platform for the Perl community. Written in Perl with a graphic design donated by Six Apart, Ltd.