Steve Bertrand

This tutorial shows how to wrap a C shared library using XS and Perl (including creating a trivial test shared library).

The first 3/4 is the actual wrapping itself. The second part includes the C code and build commands to create the trivial shared library used in this tutorial.

Relatively, I am still very new to all of this, as it's a pretty complex world. Before I started, I didn't have any real C experience, so I've been dealing with that learning curve at the same time, so I know there are better and more efficient ways of doing what I do, and would appreciate any feedback.

I'll get right to it. Here's an overview:

• find something to wrap. In this case, I've written a shared C library called xswrap (this code/info appears later in the tutorial)
• create a shell distribution that'll allow us to load our eventual XS code, which in turn has wrapped the C library
• update relevant files to make things hang together
• run into a function that can't be returned to Perl as-is, so we learn how to write our own C/XS wrapper so we can get what we need
• package it all together into a distribution

The actual C code is irrelevant at this point, but knowing the definitions in use is, so here they are for the xswrap library:

int mult (int x, int y);
void speak (const char* str);
unsigned char* arr (); // returns (0, 1, 2)

Create a new shell distribution

I use Module::Starter:

module-starter \
    --module=XS::Wrap \
    --author="Steve Bertrand" \
    --email=steveb@cpan.org \
    --license=perl

Now change into the new XS-Wrap directory, which is the root directory of the new dist. The Perl module file is located at lib/XS/Wrap.pm. I've removed a bunch of stuff for brevity, but the shell looks something like this:

package XS::Wrap;

use warnings;
use strict;

our $VERSION = '0.01';

Create the base XS file

I use Inline::C to do this for me, as like most Perl hackers, I'm often lazy. Note the flags in use. clean_after_build tells Inline to not clean up the build directory (_Inline after build). This allows us to fetch our new .xs file. name is the name of the module we're creating this XS file for.

use warnings;
use strict;

use Inline Config =>
           disable => clean_after_build =>
           name => 'XS::Wrap';
use Inline 'C';

__END__
__C__

#include <stdio.h>
#include <xswrap.h>

The resulting XS file is located in _Inline/build/XS/Wrap/Wrap.xs. Copy it to the root directory of the dist:

cp _Inline/build/XS/Wrap/Wrap.xs .

Here's what our base XS file looks like. It doesn't do anything yet, but we'll get there:

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "INLINE.h"

#include <stdio.h>
#include <xswrap.h>

MODULE = XS::Wrap  PACKAGE = main

PROTOTYPES: DISABLE

See the PACKAGE = main there? Change main to the name of our dist, XS::Wrap.

Adding the functions from the shared library to XS

Now we need to define our C functions within the XS file. After I've done that using the C definitions for the functions above, my XS file now looks like this:

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "INLINE.h"

#include <stdio.h>

MODULE = XS::Wrap  PACKAGE = XS::Wrap

PROTOTYPES: DISABLE

int mult (x, y)
    int x
    int y

void speak (str)
    const char* str

unsigned char* arr ()

Note that at this point, because we're not using Inline anymore, you can remove the include for the INLINE.h header file. However, in our case, we're going to be using some Inline functionality a bit later on, so instead of removing that, copy the INLINE.h file to the same directory we copied our XS file into: cp _Inline/build/XS/Wrap/INLINE.h .

Readying the module file for use

Now we have some work to do to pull in the XS, wrap the functions, and export them. Note that you do not need to wrap the functions with Perl, you can export them directly as depicted in the XS file if you wish, as long as you know you don't need to add any further validation or functionality before the XS imported C function is called. I'll wrap all three. The functions that each wrapped function calls is the literal C function, as advertised through the XS file we hacked above.

use warnings;
use strict;

our $VERSION = '0.01';

require XSLoader;
XSLoader::load('XS::Wrap', $VERSION);

use Exporter qw(import);

our @EXPORT_OK = qw(
    my_mult
    my_speak
    my_arr
);

our %EXPORT_TAGS;
$EXPORT_TAGS{all} = [@EXPORT_OK];

sub my_mult {
    my ($x, $y) = @_;
    return mult($x, $y);
}
sub my_speak {
    my ($str) = @_;
    speak($str);
}
sub my_arr {
    my @array = arr();
    return @array;
}

Telling the Makefile to load the external C library

Because we're using an external shared library, we need to add a directive to the Makefile.PL file. Put the following line anywhere in the Makefile.PL's `WriteMakefile() routine:

LIBS => ['-lxswrap'],

Building, installing and initial test

Let's build, install and write a test script for our new distribution.

perl Makefile.PL
make
make install

At this point, if everything works as expected, you're pretty well done. However, in the case here, we're going to unexpectedly run into some issues, and we'll need to do other things before we finalize our distribution.

Test script (example.pl). Very basic, it just tests all three wrapped functions:

use warnings;
use strict;
use feature 'say';

use XS::Wrap qw(:all);

say my_mult(5, 5);

my_speak("hello, world!\n");

my @arr = my_arr();

say $_ for @arr;

Output:

25
hello, world!

Hmmm, something is not right. The arr() C function was supposed to return an array of three elements, 0, 1, 2, but we get no output.

This is because arr() returns an unsigned char* which we can't handle correctly/directly in Perl.

In this case, I will just wrap the arr() function with a new C function (I've called it simply _arr()) that returns a real Perl array based on the output from the original C arr() function. Technically, I won't be returning anything, I'm going to just use functionality from Inline to put the list onto the stack, where Perl automatically picks it up.

To do this, I'll be leveraging Inline again, but with a couple of changes. We change the name, and add also bring in our shared library because we need it directly now.

Returning a Perl array from a C function

use warnings;
use strict;

use Inline config =>
           disable => clean_after_build =>
           name => 'Test';
use Inline ('C' => 'DATA', libs => '-lxswrap');

print "$_\n" for _arr();

 __END__
 __C__

#include <stdio.h>
#include <xswrap.h>

void _arr (){
    unsigned char* c_array = arr();

    inline_stack_vars;
    inline_stack_reset;

    int i;

    for (i=0; i<3; i++){
        inline_stack_push(sv_2mortal(newSViv(c_array[i])));
    }

    inline_stack_done;
}

After I execute that Perl script, I'm left with a new XS file within the _Inline/build/Test/Test.xs. . It looks like this:

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "INLINE.h"

#include <stdio.h>
#include <xswrap.h>

void _arr (){
    unsigned char* c_array = arr();

    inline_stack_vars;
    inline_stack_reset;

    int i;

    for (i=0; i<3; i++){
        inline_stack_push(sv_2mortal(newSViv(c_array[i])));
    }

    inline_stack_done;
}


MODULE = Test  PACKAGE = main  

PROTOTYPES: DISABLE


void
_arr ()
        PREINIT:
        I32* temp;
        PPCODE:
        temp = PL_markstack_ptr++;
        _arr();
        if (PL_markstack_ptr != temp) {
          /* truly void, because dXSARGS not invoked */
          PL_markstack_ptr = temp;
          XSRETURN_EMPTY; /* return empty stack */
        }
        /* must have used dXSARGS; list context implied */
        return; /* assume stack size is correct */

We only need a couple of pieces of it, so get out your CTRL-V and CTRL-C. Here are the sections (cleaned up a bit for brevity) that we need to copy into our real Wrap.xs file.

The C portion:

void _arr (){
    unsigned char* c_array = arr();

    inline_stack_vars;
    inline_stack_reset;

    int i;

    for (i=0; i<3; i++){
        inline_stack_push(sv_2mortal(newSViv(c_array[i])));
    }

    inline_stack_done;
}

The XS portion:

void
_arr ()
        PREINIT:
        I32* temp;
        PPCODE:
        temp = PL_markstack_ptr++;
        _arr();
        if (PL_markstack_ptr != temp) {
          PL_markstack_ptr = temp;
          XSRETURN_EMPTY;
        }
        return;

The C part goes near the top of the XS file, and the XS part goes in the XS section at the bottom. Here's our full XS file after I've merged in these changes:

Finalized XS file

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "INLINE.h"

#include <stdio.h>
#include <xswrap.h>

void _arr (){
    unsigned char* c_array = arr();

    inline_stack_vars;
    inline_stack_reset;

    int i;

    for (i=0; i<3; i++){
        inline_stack_push(sv_2mortal(newSViv(c_array[i])));
    }

    inline_stack_done;
}

MODULE = XS::Wrap  PACKAGE = XS::Wrap

PROTOTYPES: DISABLE

int mult (x, y)
    int x
    int y

void speak (str)
    const char* str

unsigned char* arr ()

void _arr ()
        PREINIT:
        I32* temp;
        PPCODE:
        temp = PL_markstack_ptr++;
        _arr();
        if (PL_markstack_ptr != temp) {
          PL_markstack_ptr = temp;
          XSRETURN_EMPTY;
        }
        return;

So, in our XS, we have four functions. Three that are imported directly from the C shared lib (mult(), speak() and arr()) and one new one written in C locally that wraps an imported XS function (_arr()).

We need to do a quick update to the wrapper in the module file. Change the call to arr() to _arr() in the .pm file within the my_arr() function:

sub my_arr {
    my @array = _arr();
    return @array;
}

Repeat the build/install steps, then test again:

perl example.pl
25
hello, world!

0
1
2

Cool! Our custom C wrapper for arr() works exactly how we want it to. We're ready for release!

Creating a release of our distribution

It's very trivial to do:

rm -rf _Inline
perl Makefile.PL
make
make test
make manifest
make install
make dist

Of course, you have written all of your POD and unit tests before reaching this point, but I digress :)

Creating and building the shared library

Here is the C code used for the above example, including build steps.

The header file (xswrap.h)

int mult (int x, int y);
void speak (const char* str);
unsigned char* arr ();

The .c file (xswrap.c)

#include <stdio.h>
#include <stdlib.h>

int mult (int x, int y){
    return x * y;
}

void speak (const char* str){
    printf("%s\n", str);
}

unsigned char* arr (){
    unsigned char* list = malloc(sizeof(unsigned char) * 3);

    int i;

    for (i=0; i<3; i++){
        list[i] = i;
    }

    return list;
}

The entry point file (main.c) for testing the lib

#include <stdio.h>
#include "xswrap.h"

int main (){
    int ret = mult(5, 5);
    printf("%d\n", ret);

    speak("hello, world!");

    unsigned char* list = arr(); 

    int i;

    for (i=0; i<3; i++){
        printf("%d\n", list[i]);
    }

    return 0;
}

The build/install script (build.sh)

#!/bin/sh
gcc -c -fPIC xswrap.c
gcc -shared -fPIC -Wl,-soname,libxswrap.so -o libxswrap.so xswrap.o -lc
sudo cp libxswrap.so /usr/lib
sudo cp xswrap.h /usr/local/include

Done!

The library and its header file are both now put into directories in your PATH.

To compile the test program:

gcc -o test main.c -lxswrap

...run it:

./test

You're now ready to wrap the library using Perl and XS.