NumPy is very well run, and very well organized, and there are many more libraries available for NumPy than there are for PDL. Those dollars are very likely to be well spent. I will not argue that this was a bad decision by DARPA.

But think about it. Do you remember just how amazing it was when Craigslist gave the Perl Foundation $100,000? Or when Booking.com gave the Perl Foundation 100,000 Euros? Now, multiply that by 30.

chromatic replied to Ovid's post by saying that we should write kick-ass software. Well, I have. I wrote a plotting library that I think is really good. It addresses one of the most important shortcomings of PDL, namely the lack of a default plotting library. It still needs a ton of work, but it fast and functional, today, and it has a strong foundation. The problem? Nobody uses the library besides me. Why? Because they already went through the labor of setting up some other plotting library, like PGPLOT, PLplot, or Gnuplot. That took a lot of effort, and climbing the learning curve took time for them. That means that the only way to really get users is to get new users. Now, tell me: where are new users coming from?

I repeat, where are new users coming from?

Potential new users ask their friends how they get their work done. Their friends say Matlab, or R, or Python. Potential new users are probably more likely to try using Ruby than they are to try using Perl because "it's cool." Everybody thinks Perl has gone the way of Tcl. And why shouldn't they? Tcl's latest major version is newer than Perl's.

What would I like? I'd like to see Perl 5.18+Moo released as Perl 7. Make it big. Make a splash. People will be wowed by the new syntax---even better than Python's or Ruby's OO sugar---and will be happy when p5mop finally hits and delivers a major speed-up.

Please, stop saying that "if you build it, they will come." We need hype.

This was my first YAPC, and it was fantastic! For many years my knowledge of the Perl community has been through the PDL mailing list, through the many Perl blogs, and through occasional IRC. I attended few Chambana.pm meetings, but they were social and didn't really get me fired up for Perl. My first experience with a collection of Perl programmers would be joining Chicago.pm last fall, and my first Perl conference was DC/Baltimore Perl Workshop this spring. But wow, 400+ Perl programmers in one place!

I gave two talks at the conference: an introduction to the Perl Data Lanuage (PDL) and an introduction to my new plotting library called PDL::Graphics::Prima. Both were well attended and well recieved, and I have gotten a handful of follow-up email and irc discussions as a result of both talks. Building the PDL community was one reason I attended YAPC::NA and I get the impression that it's paying off.

In addition to talking with recruiters, I met lots of great Perl programmers. The most anticipated was meeting Maggie Xiong, a PDL programmer who wrote the impressive and much-needed PDL::Stats distribution. She is only the second person I have met face-to-face who can write code that utilizes PDL::PP. (I met Chris Marshall, PDL pumpking, in mid-April at the DCBPW, a surprise indeed since I didn't know he would be there!) Putting a face and a voice to a personality that I knew only through email and irc was a great experience.

To my surprise, I had the opportunity to speak with Larry Wall. I called an ad-hoc meeting of scientists "in the corner by the piano" at the end of the first day's lightning talks and Larry decided to join us. My goal had been to identify that handful of us who did science (as opposed to web development :-) so that we could find each other in the hallways through the rest of the week, and that mostly worked. Larry decided to join our group because he was interested in meeting Perlers interested in discussing Synopsis 9, and that mostly worked, too. Larry and I chatted for a bit about how he wanted to incorporate PDL ideas into Perl6, and I explained my concerns about doing it "right". I have since had a handful of ideas that I've been meaning to write down and email off to Larry. Now that I've written this report, maybe I'll be able to get around to doing that.

I come away with new perspectives (which I hope to explore with more blogs) and a lot of good intentions. I hope that I can convert this energy into practical results, whether those be modules on CPAN or workshops with Chicago.pm. The first of those good intentions is finally fulfilled: writing my YAPC::NA report.

Error handling is a boring topic, but it's important, so I'm going to motivate it a bit. Consider this statement from version 4.0 of the CUDA C Best Practices Guide (which you can find here):

Code samples throughout the guide omit error checking for conciseness. Production code should, however, systematically check the error code returned by each API call...

In CUDA::Minimal, you don't have to sacrifice conciseness for error-checking...

For example, suppose you try to allocate memory on your device and you run into trouble. This error is hard to make in Perl because it allows using underscores in numbers like 10_000, but let's assume that you accidentally allocate far more memory than you meant to allocate. Here's a fully working script that should produce the error (and only the error), at least on current hardware:

use strict;
use warnings;
use CUDA::Minimal;

# Oops: 1 Terabyte of memory?
my $input_dev_ptr = Malloc( Sizeof f => 10e12);

print "I've escaped the error!\n";

Although it may seem a little contrived, this will croak with an informative message:

Unable to allocate 4294967295 bytes on the device: out of memory at cuda-error.pl line 6

Malloc can croak for a handful of reasons, but this error comes to Malloc from CUDA itself: we're asking for too much memory. The system croaks with this message. Contrast that with the CUDA-C code, which would chug merrily along unless you checked the error condition.

The simple way to handle these sorts of errors is to use eval blocks and capture errors when you know how to respond.

Thread Launch Problems

An interesting feature of CUDA is that kernel-launches are non-blocking. When you launch a kernel, as demonstrated in the opening example , control returns to the CPU immediately after the kernel starts running. This is handy because it allows you to do other things on the CPU while the calculations run on the GPU, such as logging.

However, non-blocking kernel launches cannot directly report run time errors in your kernel, such as a segmentation fault. To make matters even more confusing, kernel launch failures will trip errors when you call other functions, such as memory allocations or transfers, because the system remains in an error state until you clear it. So if you get an "Unspecified launch failure," scan backward from that point in the code to find the offending kernel launch.

To be sure you've found the correct problematic kernel launch, place the following immediately after your kernel launch:

TreadSynchronize;
croak ("Found error") if ThereAreCudaErrors;

Error-related Functions

At the moment, there are three error-related functions you should know about:

1. ThereAreCudaErrors: boolean function that returns true when an error exists and false otherwise.
2. PeekAtLastError: Returns the string describing the last error, or 'no errors'
3. GetLastError: Like PeekAtLastError, but also resets the error status.
4. DeviceReset: Resets the device so that future kernel launches do not fail from a previous "Unspecified launch failure"

First, of course, you'll need to install ExtUtils::nvcc. At the moment this only works with Linux (maybe with Mac OSX, definitely not yet with Windows). It has only been confirmed with Ubuntu. See directions on the ExtUtils::nvcc wiki. (If you manage to install it on other systems, please let me know and edit the wiki or send me your notes!) If you have that installed, installing CUDA::Minimal is just a simple CPAN install.

First Script

So, at this point I will assume you've installed CUDA::Minimal. What can you do with it? Here's a simple example:

That's not exactly hello world. Let's pull it apart.

Boiler Plate

Starting from the top, we see some fairly standard boiler-plate using strictures and warnings. Since this is an example for CUDA::Minimal, we'll need that too.

The last two lines of use statements should like a bit intriguing to you:

use ExtUtils::nvcc;
use Inline C => DATA => ExtUtils::nvcc::Inline;

ExtUtils::nvcc is part of the CUDA toolchain and it provides some simple functions for configuring the three main build tools: ExtUtils::MakeMaker, Module::Build, and as shown here Inline::C. (This sets the cc and ld flags, as explained in the docs.)

Allocating Memory

After creating a Perl string filled with a packed array of floating-point data, we come to the first lines of CUDA::Minimal:

my $input_dev_ptr = MallocFrom($host_data);

MallocFrom was imported from CUDA::Minimal by default. (Yeah, it imports functions by default. It's supposed to be easy to use. :-) MallocFrom is one of those handy functions that packs a lot of functionality compared with its CUDA C counterparts. It (1) determines the size of your host-side memory, (2) allocates the same amount of memory on the device, (3) copies the host-side data to the device, and (4) returns the pointer to the memory location on the device. All that with one function call!

The next step allocates even more memory on the device. This is the memory on the video card where the results will go:

my $output_dev_ptr = Malloc($host_data);

Malloc, is very similar to MallocFrom except that it does not copy the contents of $host_data over to the device. It simply allocates the memory and returns the device pointer.

Location and Terminology

This is a good point to introduce some terminology. CUDA provides a way for running almost arbitrary code in parallel on your video card. Video cards to not have direct access to your CPU's RAM, and your CPU does not have direct access to your video card's RAM. (nVidia's CUDA Toolkit 4.0 makes this a small lie, but stick with me.) Therefore, it is common convention to refer to the video card as the device and the CPU and its associated RAM as the host. Device pointers are often prefixed with a d_ and host pointers are commonly prefixed with a h_ to help with bookkeeping.

Although we clarify which memory is which (host vs device), we use an entirely different name for functions run on the video card. They are called kernels. We call functions on the host CPU and we launch kernels on the device.

Launching the Kernel

CUDA::Minimal does not provide a means for launching kernels directly. (Perl bindings for the so-called CUDA Driver API, which allows you to do this and many other things outside the scope of CUDA::Minimal, are my next project.) However, Perl provides a means for calling C functions using either Inline::C or plain ol' XS. If said code is compiled using nvcc (using ExtUtils::nvcc to simplify configuration), you can invoke a kernel using the CUDA-C kernel invocation syntax. I'll discuss that in a little bit. The point is that from the standpoint of Perl we are simply calling a function which happens to be defined using XS code instead of Perl code:

invoke_the_kernel($input_dev_ptr, $output_dev_ptr, $N_values);

The kernel launch is a bit of magic of which Perl is blissfully unaware.

Getting the Results and Cleaning Up

Having run the kernel, I next bring back the results. I do this by first allocating some new memory on the CPU. Perl provides a handful of methods for setting the length of scalar variables, but I can never remember them so I created a SetSize function to handle it for me. I copy the results from the video card back to this host memory and print the results:

SetSize(my $results_array, length($host_data));
Transfer($output_dev_ptr => $results_array);
print "$_\n" foreach (unpack 'f*', $results_array);

You may have noticed that I use Transfer to copy data both to and from the device. The use of the fat comma (=>) is highly recommended as it gives a very clear indication of the flow of data. Under the hood, Transfer examines the details of the scalars that you pass and determines if either or both arguments are device pointers, and takes the appropriate action. (If both are device pointers, however, you must specify the number of bytes to copy in a third, optional argument.) Finally, we free the device-side memory:

Free($input_dev_ptr, $output_dev_ptr);

Your program will execute fine without freeing the memory and the memory will (as best I can tell) be reclaimed by the video card at the close of your program. However, if you have a long-running script, failure to free device memory may lead to allocation issues, so it is a good practice to free device-side memory when you're done with it.

The Kernel Definition

CUDA kernels are defined like normal functions in C with one special addition: the use of __global__ before the return value:

__global__ void triple(float * in_g, float * out_g) {
    out_g[threadIdx.x] = in_g[threadIdx.x] * 3;
}

Furthermore, all kernels have access to the variables threadIdx, blockIdx, blockDim, and gridDim, though I will not explain those now. Inline::C that does not recognize such statements as normal C function declarations, which is very important for the use of Inline::C with CUDA. (It is likely a bug, but it sure is useful.)

The Kernel Wrapper

As I said earlier, Perl does not (yet) have wrappers for API that would allow for direct kernel invocation. However, Inline::C knows how to parse a standard function definition and expose that function to Perl for me. The role of the C function is then to unpack the arguments and invoke the kernel:

void invoke_the_kernel(SV * in_SV, SV * out_SV, int N_values) {
    // Unpack the device pointers:
    float * d_in = INT2PTR(float *, SvIV(in_SV));
    float * d_out = INT2PTR(float *, SvIV(out_SV));

    // invoke the kernel:
    triple <<<1, N_values>>>(d_in, d_out);
}

The first two lines of this function convert Perl's internal representation of the input scalars into native C float pointers. These pointers point to the location on the video card where the data resides and are passed along to the kernel so it knows from where it retrieves its input and stores its results.

The last and most interesting part of this whole chunk of code is the means by which we invoke the kernel. func-name <<< N-blocks, block-size >>> ( args ). If you strip away the parts between the triple-brackets, we have something that looks like a normal function call. The triple-brackets are an extension to ANSI-C provided by nvcc and is one reason that CUDA code must be compiled with nvcc: when nvcc finds triple-brackets, it inserts code to initialize the video card to run the kernel with the associated block dimensions and grid dimensions.

Summary

Today I've given a simple Perl script that manages CUDA memory, compiles a CUDA kernel, and invokes the CUDA kernel with a very thin wrapper written in C. Tomorrow I'll discuss CUDA interoperability with PDL.