C::TinyCompiler, a just-in-time C compiler for Perl

There are many things you might do to speed up your Perl code. After profiling and benchmarking, you revise the slow spots, or maybe try a different algorithm. If you still need more speed, you may use Inline::C, or use PDL if you are doing numerics, or even write your own XS code. Reini Urban has even tried to create a Perl Backend that rewrites your Perl code as C code, and has even worked on a port of Perl5 to Potion.

But let's face it. Sometimes it would just be easier if you could write your code in C, and interpolate a string into it. Wouldn't it be nice if there was a C equivalent of Perl's string eval?

Well, in Perl, now there is.

First, some background. The Tiny C Compiler is a small, fast, nearly C99 compliant C compiler written for Linux and Windows, which can also compile object code on Mac (but cannot produce Mach-O executables or dylibs). It targets 32- and 64-bit Intel instruction sets as well as ARM architectures. The compiler is very fast, but what you get in compile-time speed you lose in non-optimized machine code. However, it's still machine code, and it offers a library that includes functionality for compiling arbitrary strings of C code at runtime!

C::TinyCompiler provides an interface to using that just-in-time compilation capability, as well as a framework for building reusable components in the system. This provides the string eval for C code. :-)

The reason I am writing about this today is because I just pushed the firs version of Alien::TinyCC to CPAN. Alien::TinyCC installs the Tiny C Compiler to a non-invasive location which is almost certainly not in the user's path, but the module itself fiddles with $ENV{PATH} and, possibly, $ENV{LD_LIBRARY_PATH} to ensure that the compiler and its main library are available within the Perl environment that invokes the module. Note that this module does not pay any attention to a local install of tinycc, it always installs tcc under the distribution sharedir for Alien::TinyCC.

What does this mean? If you are using one of the major platforms (Window, Mac, Linux) and have to generate code at runtime, try

cpanm C::TinyCompiler

It should Just Install (at least, once Alien::TinyCC 0.02 reaches your local cpan mirror). Then check out C::TinyCompiler::Callable so you can learn how to invoke your C functions from your Perl code, C::TinyCompiler::Perl::Croak so you can throw Perl exceptions in your C code, and C::TinyCompiler::StretchyBuffer so you can push and pop on C arrays as if they were Perl arrays.

The library is still in the early stages of development, and I'd love feedback and ideas. I'm sure the Alien package breaks a few rules here and there (on Unixen it directly installs tcc into the sharedir, for example, which probably breaks some indexing system somewhere) and the code underlying C::TinyCompiler::Callable could be substantially streamlined, I'm sure. But I figured it was important to get the thing out ther and let folks play with it.

Let me close with this simple example interfacing PDL data with a function written using C::TinyCompiler:

 use strict;
 use warnings;
 use PDL;
 use C::TinyCompiler;
 my $context = C::TinyCompiler->new('::Callable');
 # Write a function to create a sequence of prime numbers:
 $context->code('Body') = q{
     void prime_sequence (int * output, int length) {
         /* Always start with 2 */
         output[0] = 2;
         int n_filled = 1;
         int candidate = 3;
         while(n_filled < length) {
             for (int divisor_idx = 0; divisor_idx < n_filled; divisor_idx++) {
                 if (candidate % output[divisor_idx] == 0) goto NEXT_NUMBER;
                 if (output[divisor_idx] * output[divisor_idx] > candidate) break;
             output[n_filled] = candidate;
             NEXT_NUMBER: candidate++;
 # Compile our C code
 # Retrieve a subref to our function
 my $prime_sequence = $context->get_callable_subref('prime_sequence');
 # Allocate some memory for the operation
 use PDL;
 my $primes = zeroes(long, 20);
 # Exercise the subref to create the first 20 primes
 $prime_sequence->($primes->get_dataref, $primes->nelem);
 print "First 20 primes are $primes\n";



I had to do a

cpanm Alien::TinyCC --dev

for it to work on Strawberry. Overall an awesome package. Keep up the awesome work.

Why is it not on GitHub? :)

Looks nice and clean.
So the remaining sugar is needed as for every other FFI like package:
Handle foreign and perl primitive and aggregate data structures more easily (where ctypes got stuck).
Apparently I have to lookup PDL, which seems to provide zeroes.

The name 'get_callable_subref' is a bit too long for my taste. How about 'call' or 'func' or 'sym'?

BTW: perlcc is used by cPanel in production for about 15 years, and is very stable.
Not all perl5 features are supported, but all which are needed for modern perl and 99% of CPAN

Leave a comment

About David Mertens

user-pic This is my blog about numerical computing with Perl.