doubi

Hello, good evening and welcome.

For the next few months I will be using this blog to help document and publicise my "Ctypes for Perl" project. The project is being carried out for TPF under the auspices of the Google Summer of Code programme, and mentored by Reini Urban.

What's a ctypes?

'ctypes' is the Foreign Function Interface (FFI) library distributed with the Python core. It basically allows native C libraries to be called easily from Python; for module authors, it allows the wrapping of C libraries in pure Python.

This is obviously a powerful concept. Imagine a world where Perl module authors didn't need to use XS, and module consumers don't need to have a correctly configured compiler set up on their system. This is the purpose of the project: to create an easy, cross-platform, pure-Perl interface to native C libraries.

Implementations

ctypes is based on libffi. It's small, supports a wide range of systems, and has a very liberal license. It's been distributed with GCC for a number of years, used by gcj for interfacing between interpreted and compiled code.

From what I can gather, Python set the trend in dynamic languages using libffi. Looking at the success of the Python module, developers at Mozilla chose libffi to develop ctypes.jsm. Ruby-FFI uses it too, so there's plenty of prior art which will hopefully help me out.

The FFI problem hasn't been ignored in the Perl world. There's FFI.pm, the biggest disadvantage of which in my view is being built on libffcall, a library analogous to libffi but under the GPL (I don't think libffi was around at the time FFI.pm was written). It also sets out to provide a 'low-level' interface. P5NCI, on the other hand, is all about the lovely interfaces, but only allows up to four arguments passed to C functions, and doesn't yet support passing in pointers. C::Dynalib provides similar functionality to the other two modules; click here for the latest updates on its development. It's worth pointing out that none of these modules worked out of the box on Strawberry 5.10.1.

My proposed API rolls in features of several of the above implementations, particularly P5NCI and FFI.pm. I have indeed copied and pasted swathes from their POD pages (So what? Wanna fight about it?). I plan to also mimic C::DynaLib's acceptance of both positional & named parameters; examples are omitted below for succinctness.

1. Functional

    use Ptypes;
    # FFI.pm's interface of Absolute Freedom...
    my $addr = (address of a C function)
    my $signature = (function signature)
    my $ret = Ptypes::call($addr, $signature, ...);
    
    # Keeping things where you can see them...
    my $library_path = Ptypes::find_lib( 'mathtastic' );
    my $library      = Ptypes::load_lib( $library_path );
    my $double_func  = Ptypes::load_func( $library,
                                          'double_double',
                                          'dd' );
    my $ret   = $double_func->( 1.0 );

    # Supplying a Perl callback...
    $ret = Ptypes::call($addr, $signature, $subref, ...)

2. Objectionable

    use Ptypes;
    my $lib = Ptypes->new( library => 'mathtastic',
                         [ package => 'MyPackage' ] );
    my $double_func = $lib->load_function( 'double_double', 
                                           'dd' );
    my $ret  = $double_func->( 1.0 );

    # Exposing funcs directly in Perl namespaces...
    $lib->install_function( 'double_int',
                            'ii',
                            [ 'perl_sub_name',]
                            [ 'AnotherPackage' ] );
    my $ret = AnotherPackage::double_int( 1 );
    # or simply...
    package AnotherPackage;
    my $ret = double_int( 3 );

All fairly self-explanatory, perhaps apart from arguments like 'ii' or 'dd' - these strings describe return values and arguments for C functions in the same notation used by pack. In addition to the above, the module may provide mechanisms for manipulating C data types directly from Perl ($c = new Ptypes::char). To start off with though there'll be a fairly straight-forward / 'stupid' translation based on seeing what kind of data's in your SV*, for initial experimentation.

There's also some exciting stuff to do with GCC::TranslationUnit to tell you about, but details of that will wait till later. For now, I have some questions for you, the community:

• How d'you like the API proposal above? Anything you'd add? Take out?
• How does 'Ptypes' take you as a name for this malarky? Y'know, like ctypes, but for Perl. 'FFI' is already taken after all...

Don't want to gush here, but I'm so chuffed* to be working on this. I'm already learning loads, and I think it will save a lot of blood, sweat & tears for module authors and users in the future. I want to thank rurban for his guidance & help so far, and dukeleto and others for organising the The Perl Foundation's participation in GSoC and letting me participate!

I like to work log, so follow @doubious_code on Twitter to get far more information than you want about this project. I hope to be blogging pretty regularly too.

* For American-speakers, 'chuffed' is kinda equivalent to 'stoked'