Graham Ollis

If you've heard of FFI::Platypus you probably know that you can use it to easily write Perl bindings for existing system libraries:

use FFI::Platypus 1.00;
use FFI::CheckLib qw( find_lib_or_die );
my $ffi = FFI::Platypus->new( api => 1 );
# uses the system libarchive
$ffi->lib( find_lib_or_die( lib => 'archive' ) );

You may even know that Platypus integrates nicely with Alien for seamless installation experience, even if the library isn't already installed:

use FFI::Platypus 1.00;
use Alien::Libarchive3;
my $ffi = FFI::Platypus->new( api => 1 );
$ffi->lib( Alien::Libarchive3->dynamic_libs );

What you might not realize is that you can also bundle your own compiled code with your Perl extension. There are a couple of reasons that you might want to do this. For one you might have a tight loop in your Perl application that would benefit from the speed that a compiled language would give you. For another, you might need to write a little code to smooth over the edges of an FFI unfriendly library. C++ for example is quite unfriendly to FFI if it doesn't provide a C layer, and this allows you to bundle your own C layer. Whatever the case, all you need to do is throw some C code into a directory named ffi. Lets use this simple person_t class written in C.

/* lives in ffi/person.c */

#include <stdlib.h>
#include <string.h>

typedef struct {
    char *name;
    int lucky_number;
} person_t;

person_t *
person_new(const char *class, char *name, int lucky_number) {
    person_t *self;
    self = calloc(1, sizeof(person_t));
    self->name = strdup(name);
    self->lucky_number = lucky_number;
    return self;
}

char *
person_name(person_t *self) {
    return self->name;
}

int
person_lucky_number(person_t *self) {
    return self->lucky_number;
}

void
person_DESTROY(person_t *self) {
    free(self->name);
    free(self);
}

Now we can write a Perl class based on this C code:

package Person;

use strict;
use warnings;
use 5.014;
use FFI::Platypus 1.00;

my $ffi = FFI::Platypus->new( api => 1 );

# use the bundled code as a library
$ffi->bundle;

# use the person_ prefix
$ffi->mangler(sub {
    my $symbol = shift;
    return "person_$symbol";
});

# Create a custom type mapping for the person_t (C) and Person (perl)
# classes.
$ffi->type( 'object(Person)' => 'person_t' );

$ffi->attach( new          => [ 'string', 'string', 'int' ] => 'person_t' );
$ffi->attach( name         => [ 'person_t' ] => 'string' );
$ffi->attach( lucky_number => [ 'person_t' ] => 'int' );
$ffi->attach( DESTROY      => [ 'person_t' ] );

1;

A few things here are worth noticing. First of all we use the bundle method to find the C code that goes with this class.

$ffi->bundle

While we are developing this class it will:

1. Find the code in the ffi directory,
2. Compile it using the C compiler
3. Link into a dynamic library
4. Configure Platypus to use that dynamic library.

Later on, I will show how we can use ExtUtils::MakeMaker to build this code at install time just once and use it at run time, so it will:

1. Find the dynamlic library already installed in the appropriate location
2. Configure Platypus to use that

Because the code is compiled just once at install time, you don't need to have a compiler on your production system, just on your development and build systems. This makes the prerequisites for your distribution no more onerous than bundling code with XS. (Arguably its easier actually because you don't have to build your bundled code as a separate step with Platypus in develop mode).

Under the covers both development and install time builds use FFI::Build to compile and link the dynamic library as necessary. It's a modular system which can support other languages, like Fortran, Rust or even Go. This means that you can write your Perl extension in languages other than C, and leverage the available code libraries in a wide variety of programming languages. FFI::Build is even smart enough to check time stamps on the libraries and source code so that if the source code hasn't changed they won't be re-built.

Side note: you can use both the bundle method and the lib attribute with the same Platypus instance. This can be helpful when dealing with an FFI-unfriendly library that requires a little C code to work with FFI, extracting constants from a header file (I will discuss this in another entry), or if you just need to write some auxiliary C code to go along with your bindings.

$ffi->lib( find_lib_or_die(lib => "foo") );  # finds libfoo.so or local platform equivalent
$ffi->bundle;                                # finds or builds the bundled dynamic library

Next we use the mangler method to tell Platypus that all the functions that we are interested in start with the person_ prefix

$ffi->mangler(sub {
    my $symbol = shift;
    return "person_$symbol";
});

This is a nice short-cut if you have lots of methods that will save you some typing. More importantly it is also handy because for our Perl class we will want to attach the method names without the person_ prefix, since the methods will already be scoped inside the Person class. Without this little trick you'd have to specify both the C and the Perl names in your call to attach:

$ffi->attach( [ person_new => 'new' ] => [ 'string', 'string', 'int' ] => 'person_t' );

For the last step before defining our methods we need to define a custom type to map between C and Perl space for our Person class:

$ffi->type( 'object(Person)' => 'person_t' );

The object type is new to Platypus as of 1.00 and requires the version 1 API be enabled. It makes it easy to write bindings for libraries that use opaque pointers as object references. It's a common pattern as demonstrated by the person_t class above. In Perl space the object is stored as a reference to an opaque pointer.

The last part of the module just creates the Person methods using the Platypus attach method.

$ffi->attach( new          => [ 'string', 'string', 'int' ] => 'person_t' );
$ffi->attach( name         => [ 'person_t' ] => 'string' );
$ffi->attach( lucky_number => [ 'person_t' ] => 'int' );
$ffi->attach( DESTROY      => [ 'person_t' ] );

We've written the C interface so that it works nicely as a Perl class. If you are using this for an existing library that doesn't have such a Perl friendly interface you can use function wrappers to smooth over the differences. For example, often the new function of a C class like this won't take the class name as a first argument (why would it?). If our person_t class didn't take (and ignore) the class name we could write the wrapper for that function like this:

person_t * person_new(char *name, int lucky_number);

$ffi->attach( new => [ 'string', 'int' ] => 'person_t' => sub {
    # ignore the class name.
    my($xsub, undef, $name, $lucky_number) = @_;
    $xsub->($name, $lucky_number);
});

Now we could use this from our Perl script, but lets write a test first:

use Test2::V0;
use Person;

my $plicease = Person->new("Graham Ollis", 42);

is $plicease->name, "Graham Ollis";
is $plicease->lucky_number, 42;

done_testing;

Notice that the user of the Person class in Perl doesn't need to know or care that the underlying implementation is in C or uses FFI. The test just works:

veracious% perl -Ilib t/basic.t 
# Seeded srand with seed '20191121' from local date.
ok 1
ok 2
1..2

Finally, I promised to show you how to write your Makefile.PL so that you can bundle your code with your distribution. There is a helper class FFI::Build::MM for that!

use ExtUtils::MakeMaker;
use FFI::Build::MM;

my $fbmm = FFI::Build::MM->new;

WriteMakefile($fbmm->mm_args(
    ABSTRACT       => 'My Person class',
    DISTNAME       => 'Person',
    NAME           => 'Person',
    VERSION_FROM   => 'lib/Person.pm',
    CONFIGURE_REQUIRES => {
        'FFI::Build::MM'          => '1.00',
    },
    BUILD_REQUIRES => {
        'FFI::Build::MM'          => '1.00',
    },
    PREREQ_PM => {
        'FFI::Platypus'           => '1.00',
    },
    TEST_REQUIRES => {
        'Test2::V0' => '0',
    },
));

sub MY::postamble {
     $fbmm->mm_postamble;
}

$ perl Makefile.PL 
Generating a Unix-style Makefile
Writing Makefile for Person
Writing MYMETA.yml and MYMETA.json
$ make
cp lib/Person.pm blib/lib/Person.pm
"/usr/bin/perl" -MFFI::Build::MM=cmd -e fbx_build
CC ffi/person.c
LD blib/lib/auto/share/dist/Person/lib/libPerson.so
$ make test
"/usr/bin/perl" -MFFI::Build::MM=cmd -e fbx_build
"/usr/bin/perl" -MFFI::Build::MM=cmd -e fbx_test
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/basic.t .. ok   
All tests successful.
Files=1, Tests=2,  0 wallclock secs ( 0.01 usr  0.00 sys +  0.07 cusr  0.00 csys =  0.08 CPU)
Result: PASS

If you are using Dist::Zilla, there is a plugin for that: Dist::Zilla::Plugin::FFI::Build. Just add this to your dist.ini:

; even easier!
[FFI::Build]

We've learned today that we can bundle C code with our Perl distribution using FFI instead of XS. This allows us to implement part of our Perl application in C, or to interface more easily with FFI-unfriendly libraries. We also saw the new Platypus object type in action, and how it works with opaque pointer objects in C. I mentioned that the same system that lets you bundle C code, is extendable and can be used to bundle other compiled languages, like Fortran, Rust or Go, which we will demonstrate in the future.