What would Psychic::Ninja do?

I have the Psychic::Ninja namespace. I was making a new GitHub repository, and GitHub suggested that name. I took it, but I had no idea what to do with it. I created a distribution that doesn't do anything.

I've been thinking about what a psychic ninja would actually do. He'd be a highly skilled worker who gets your work done without you knowing he did it, and he'd do it before you knew you wanted it done. If a psychic ninja was your gardener, you'd never know that grass actually grew, but you'd have the best lawn in the neighborhood.

How would that sort of thing show up in Perl? I haven't had many ideas about that yet, but I've been slowly hatching some idea where Perl can recognize structured data. We can play with XML, JSON, YAML, and all sorts of other formats, but we do that by loading a particular module and telling perl to use that module to parse the data we give it.

A bit more sophisticated are module families. We have several parsers for each of those types, and some *::Any modules to select the best one that's installed. Can we go ten steps further to make an Any::Any? Something detects the incoming data, knows the module family, and loads that. This technology already exists. It's File::Magic, which is just a Perl implementation of something we can do on the command line. I'm not talking about cheating with MIME types either. That's just Ninja. I have Psychic::Ninja.

The output side would be similar, but that's more complicated. If you start outputting HTML to a socket, it adds CGI or HTTP headers. If you output something that looks like CSV, you get an Excel spreadsheet. If you output XML, it directs it to /dev/null. Perhaps it looks at the directory you're in or where you are putting the data to figure out what you likely want for the output format. Some of this technology already exists in IO::All.

Those two things I can imagine having lots of smarts and heuristics. We can train our psychic ninja to do all that. It's the stuff in the middle that I need to figure out. Is there enough information in the universe that the module can make a good guess at the task? Besides all the machine learning stuff to draw from previous runs, what other things can the psychic ninja observe? Command history? The rest of the command line, including previous and future commands in the pipeline? That is, can I just do this and have the right thing happen?

% pninja < input.xml > output.xls

It doesn't really matter what the answer is because it will end up on my list of all the other things I think about and do nothing with. I do like thinking about how much I can reduce user instructions to a program and still get what I want.

It might be easier to figure out what an Glowing Octo Ironman would do. That's just a triathlete who's drank too much Red Bull and happens to have extra arms and legs.


Not dress in black.

I think the output format sniffing is too ambitious, and may be counterproductive since knowing both input and output types could make the program much smarter. For example, if you know your input is HTML and your output is CSV, you can assume you're dealing with tabular data, and just pull the tables. The equivalent of perl -pe or -Ane with these extra heuristics could be useful.

Personally, I would be happy with something that tried to do for text what "convert" does for images, with a bit of optional Perl in the middle.

Leave a comment

About brian d foy

user-pic I'm the author of Mastering Perl, and the co-author of Learning Perl (6th Edition), Intermediate Perl, Programming Perl (4th Edition) and Effective Perl Programming (2nd Edition).