#define
pre-processor directives which are lost by the time the code is linked into a dynamic library. (In fact it is lost before the source is even compiled, since it is a pre-processor directive). For example, the libarchive library provides these constants in its header file for dealing with errors:
/*
* Error codes: Use archive_errno() and archive_error_string()
* to retrieve details. Unless specified otherwise, all functions
* that return 'int' use these codes.
*/
#define ARCHIVE_EOF 1 /* Found end of archive. */
#define ARCHIVE_OK 0 /* Operation was successful. */
#define ARCHIVE_RETRY (-10) /* Retry might succeed. */
#define ARCHIVE_WARN (-20) /* Partial success. */
/* For example, if write_header "fails", then you can't push data. */
#define ARCHIVE_FAILED (-25) /* Current operation cannot complete. */
/* But if write_header is "fatal," then this archive is dead and useless. */
#define ARCHIVE_FATAL (-30) /* No more operations are possible. */
(I'll also discuss enumerated types (enums), which are in some ways similar, but in some ways provide additional challenges).
This is especially a challenge for writing FFI bindings in dynamic languages like Perl using FFI::Platypus. If there are only a few constants, and if they are relatively stable (as is the case with libarchive), then it might be easiest to just define equivalent values in Perl space with the same names:
package Archive::Libarchive::FFI;
use strict;
use warnings;
use constant ARCHIVE_EOF => 1;
use constant ARCHIVE_OK => 0;
use constant ARCHIVE_RETRY => -10;
use constant ARCHIVE_WARN => -20;
use constant ARCHIVE_FAILED => -25;
use constant ARCHIVE_FATAL => -30;
...
This starts to break down when there are hundreds of constants, or if they change from version to version, or if they depend on system defined constants (like the maximum length of a file path, or the size of a long int
). One way to deal with this would be to bundle some C code with your distribution (See Bundling With Platypus) with functions that provide the name/value mappings:
#include <archive.h>
int const_archive_ok() { return ARCHIVE_OK; }
int const_archive_eof() { return ARCHIVE_EOF; }
...
This adds a lot of overhead though, for something that will never change. Each constant has to have its own function, and the return value never changes. There will be a lot of repetition in the corresponding .pm file to create the bindings to each constant function. It would be nice if you could define all the constants that are needed in Perl space from C space in one fell swoop. Fortunately the Platypus bundle interface provides a mechanism for doing exactly that!
/* lives in ffi/constant.c */
#include <ffi_platypus_bundle.h>
#include <archive.h>
void
ffi_pl_bundle_constant(const char *package, ffi_platypus_constant_t *c)
{
c->set_sint("ARCHIVE_EOF", ARCHIVE_EOF);
c->set_sint("ARCHIVE_OK", ARCHIVE_OK);
c->set_sint("ARCHIVE_RETRY", ARCHIVE_RETRY);
c->set_sint("ARCHIVE_WARN", ARCHIVE_WARN);
c->set_sint("ARCHIVE_FAILED", ARCHIVE_FAILED);
c->set_sint("ARCHIVE_FATAL", ARCHIVE_FATAL);
}
The API for the ffi_platypus_const_t
object lets you define different types of constants. See FFI::Platypus::Constant for more details. (Floating point, string, signed and unsigned integer types are supported). The Perl code that goes along with this doesn't have to do anything special except for to call $ffi->bundle
, because Platypus knows to call ffi_pl_bundle_constant
if it has been defined. This means that if you need to add or modify constants you only need to do it in one place:
package Archive::Libarchive::FFI {
use strict;
use warnings;
use FFI::Platypus 1.00;
use base qw( Exporter );
my $ffi = FFI::Platypus->new( api => 1 );
$ffi->find_lib( lib => 'archive' );
$ffi->bundle;
# use this trick to export all constants with the ARCHIVE_ prefix:
our @EXPORT = grep /^ARCHIVE_/, keys %Archive::Libarchive::FFI::
...
}
What about enumerated types (enums)? The names for enum values work essentially the same as constants from an FFI perspective. They are lost when you create a dynamic library, so you can use either method for defining the values in your bundled C code or in your .pm file. What type you should use may not be so clear. Enums are usually represented as an integer type of some kind, but the C standard doesn't say exactly how big or if it should be signed. (perhaps obviously enums with negative values should be stored as signed integers, but for enums that happen to have only positive values, it is not as clear).
If you knew that enums were stored as unsigned 32bit integers on your platform, you could use sint32
. Your bindings wouldn't be portable to other platforms though, so don't do that!
To save you some work, Platypus probes the C compiler at install time to see what actual integer types are used for "signed" and "unsigned" enums. Thus, if libarchive had chosen an enum instead of #define
constants for its error code, its header file might looks something like this:
enum archive_error_code_t {
ARCHIVE_EOF = 1,
ARCHIVE_OK = 0,
ARCHIVE_RETRY = -10,
ARCHIVE_WARN = -20,
ARCHIVE_FAILED = -25,
ARCHIVE_FATAL = -30,
}
From Perl / Platypus you can define the values the same way as before, but the actual type you should use is senum
, since the possible values include negative numbers. If they were all zero or positive, then we would use enum
.
use FFI::Platypus 1.00;
my $ffi = FFI::Platypus->new( api => 1 );
$ffi->type( 'senum' => 'archive_error_code_t' );
This type alias can be used for any function that returns an error in this way.
We've learned today that there are a couple of ways to get constants out of C header files into your Perl program using FFI. Ideally there aren't a lot of magic numbers like this in whatever library you are adapting and maybe you can just put them in your .pm file as constants. If not then there is a nifty bundle interface that lets you define constants in Perl space from C space. We also learned that enums are conceptually quite similar to constants, but care needs to be taken about which types you use if you want your program to be portable.
]]>use FFI::Platypus 1.00;
use FFI::CheckLib qw( find_lib_or_die );
my $ffi = FFI::Platypus->new( api => 1 );
# uses the system libarchive
$ffi->lib( find_lib_or_die( lib => 'archive' ) );
You may even know that Platypus integrates nicely with Alien for seamless installation experience, even if the library isn't already installed:
use FFI::Platypus 1.00;
use Alien::Libarchive3;
my $ffi = FFI::Platypus->new( api => 1 );
$ffi->lib( Alien::Libarchive3->dynamic_libs );
What you might not realize is that you can also bundle your own compiled code with your Perl extension. There are a couple of reasons that you might want to do this. For one you might have a tight loop in your Perl application that would benefit from the speed that a compiled language would give you. For another, you might need to write a little code to smooth over the edges of an FFI unfriendly library. C++ for example is quite unfriendly to FFI if it doesn't provide a C layer, and this allows you to bundle your own C layer. Whatever the case, all you need to do is throw some C code into a directory named ffi
. Lets use this simple person_t
class written in C.
/* lives in ffi/person.c */
#include <stdlib.h>
#include <string.h>
typedef struct {
char *name;
int lucky_number;
} person_t;
person_t *
person_new(const char *class, char *name, int lucky_number) {
person_t *self;
self = calloc(1, sizeof(person_t));
self->name = strdup(name);
self->lucky_number = lucky_number;
return self;
}
char *
person_name(person_t *self) {
return self->name;
}
int
person_lucky_number(person_t *self) {
return self->lucky_number;
}
void
person_DESTROY(person_t *self) {
free(self->name);
free(self);
}
Now we can write a Perl class based on this C code:
package Person;
use strict;
use warnings;
use 5.014;
use FFI::Platypus 1.00;
my $ffi = FFI::Platypus->new( api => 1 );
# use the bundled code as a library
$ffi->bundle;
# use the person_ prefix
$ffi->mangler(sub {
my $symbol = shift;
return "person_$symbol";
});
# Create a custom type mapping for the person_t (C) and Person (perl)
# classes.
$ffi->type( 'object(Person)' => 'person_t' );
$ffi->attach( new => [ 'string', 'string', 'int' ] => 'person_t' );
$ffi->attach( name => [ 'person_t' ] => 'string' );
$ffi->attach( lucky_number => [ 'person_t' ] => 'int' );
$ffi->attach( DESTROY => [ 'person_t' ] );
1;
A few things here are worth noticing. First of all we use the bundle
method to find the C code that goes with this class.
$ffi->bundle
While we are developing this class it will:
ffi
directory,Later on, I will show how we can use ExtUtils::MakeMaker
to build this code at install time just once and use it at run time, so it will:
Because the code is compiled just once at install time, you don't need to have a compiler on your production system, just on your development and build systems. This makes the prerequisites for your distribution no more onerous than bundling code with XS. (Arguably its easier actually because you don't have to build your bundled code as a separate step with Platypus in develop mode).
Under the covers both development and install time builds use FFI::Build
to compile and link the dynamic library as necessary. It's a modular system which can support other languages, like Fortran, Rust or even Go. This means that you can write your Perl extension in languages other than C, and leverage the available code libraries in a wide variety of programming languages. FFI::Build
is even smart enough to check time stamps on the libraries and source code so that if the source code hasn't changed they won't be re-built.
Side note: you can use both the bundle
method and the lib
attribute with the same Platypus instance. This can be helpful when dealing with an FFI-unfriendly library that requires a little C code to work with FFI, extracting constants from a header file (I will discuss this in another entry), or if you just need to write some auxiliary C code to go along with your bindings.
$ffi->lib( find_lib_or_die(lib => "foo") ); # finds libfoo.so or local platform equivalent
$ffi->bundle; # finds or builds the bundled dynamic library
Next we use the mangler
method to tell Platypus that all the functions that we are interested in start with the person_
prefix
$ffi->mangler(sub {
my $symbol = shift;
return "person_$symbol";
});
This is a nice short-cut if you have lots of methods that will save you some typing. More importantly it is also handy because for our Perl class we will want to attach the method names without the person_
prefix, since the methods will already be scoped inside the Person
class. Without this little trick you'd have to specify both the C and the Perl names in your call to attach:
$ffi->attach( [ person_new => 'new' ] => [ 'string', 'string', 'int' ] => 'person_t' );
For the last step before defining our methods we need to define a custom type to map between C and Perl space for our Person class:
$ffi->type( 'object(Person)' => 'person_t' );
The object
type is new to Platypus as of 1.00 and requires the version 1 API be enabled. It makes it easy to write bindings for libraries that use opaque pointers as object references. It's a common pattern as demonstrated by the person_t
class above. In Perl space the object is stored as a reference to an opaque pointer.
The last part of the module just creates the Person methods using the Platypus attach
method.
$ffi->attach( new => [ 'string', 'string', 'int' ] => 'person_t' );
$ffi->attach( name => [ 'person_t' ] => 'string' );
$ffi->attach( lucky_number => [ 'person_t' ] => 'int' );
$ffi->attach( DESTROY => [ 'person_t' ] );
We've written the C interface so that it works nicely as a Perl class. If you are using this for an existing library that doesn't have such a Perl friendly interface you can use function wrappers to smooth over the differences. For example, often the new
function of a C class like this won't take the class name as a first argument (why would it?). If our person_t
class didn't take (and ignore) the class name we could write the wrapper for that function like this:
person_t * person_new(char *name, int lucky_number);
$ffi->attach( new => [ 'string', 'int' ] => 'person_t' => sub {
# ignore the class name.
my($xsub, undef, $name, $lucky_number) = @_;
$xsub->($name, $lucky_number);
});
Now we could use this from our Perl script, but lets write a test first:
use Test2::V0;
use Person;
my $plicease = Person->new("Graham Ollis", 42);
is $plicease->name, "Graham Ollis";
is $plicease->lucky_number, 42;
done_testing;
Notice that the user of the Person
class in Perl doesn't need to know or care that the underlying implementation is in C or uses FFI. The test just works:
veracious% perl -Ilib t/basic.t
# Seeded srand with seed '20191121' from local date.
ok 1
ok 2
1..2
Finally, I promised to show you how to write your Makefile.PL
so that you can bundle your code with your distribution. There is a helper class FFI::Build::MM
for that!
use ExtUtils::MakeMaker;
use FFI::Build::MM;
my $fbmm = FFI::Build::MM->new;
WriteMakefile($fbmm->mm_args(
ABSTRACT => 'My Person class',
DISTNAME => 'Person',
NAME => 'Person',
VERSION_FROM => 'lib/Person.pm',
CONFIGURE_REQUIRES => {
'FFI::Build::MM' => '1.00',
},
BUILD_REQUIRES => {
'FFI::Build::MM' => '1.00',
},
PREREQ_PM => {
'FFI::Platypus' => '1.00',
},
TEST_REQUIRES => {
'Test2::V0' => '0',
},
));
sub MY::postamble {
$fbmm->mm_postamble;
}
$ perl Makefile.PL
Generating a Unix-style Makefile
Writing Makefile for Person
Writing MYMETA.yml and MYMETA.json
$ make
cp lib/Person.pm blib/lib/Person.pm
"/usr/bin/perl" -MFFI::Build::MM=cmd -e fbx_build
CC ffi/person.c
LD blib/lib/auto/share/dist/Person/lib/libPerson.so
$ make test
"/usr/bin/perl" -MFFI::Build::MM=cmd -e fbx_build
"/usr/bin/perl" -MFFI::Build::MM=cmd -e fbx_test
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/basic.t .. ok
All tests successful.
Files=1, Tests=2, 0 wallclock secs ( 0.01 usr 0.00 sys + 0.07 cusr 0.00 csys = 0.08 CPU)
Result: PASS
If you are using Dist::Zilla
, there is a plugin for that: Dist::Zilla::Plugin::FFI::Build
. Just add this to your dist.ini:
; even easier!
[FFI::Build]
We've learned today that we can bundle C code with our Perl distribution using FFI instead of XS. This allows us to implement part of our Perl application in C, or to interface more easily with FFI-unfriendly libraries. We also saw the new Platypus object
type in action, and how it works with opaque pointer objects in C. I mentioned that the same system that lets you bundle C code, is extendable and can be used to bundle other compiled languages, like Fortran, Rust or Go, which we will demonstrate in the future.
# old code:
use FFI::Platypus;
my $ffi = FFI::Platypus->new;
# new code:
use FFI::Platypus 1.00;
my $ffi = FFI::Platypus->new( api => 1 );
You should generally write all new code using the new API so that you can take advantage of the newer features and design fixes. You may want to also consider upgrading your existing code to use the new API for the same reasons.
The main enhancement you get is the ability to pass and return records by value in addition to as pointers of by reference. This missing feature was a design bug in the original implementation of Platypus, and I am pleased to make this feature available. In the old interface records are assumed to be pointers. In the new interface they are assumed to be records, and you have to use the pointer decorator (*
) to use pointers to records.
# both versions:
package Foo {
use FFI::Platypus::Record;
record_layout
int bar
int baz
;
}
# old code:
use FFI::Platypus;
my $ffi = FFI::Platypus->new;
# XXX can't use Foo pass-by-value # pass-by-value
$ffi->type('record(Foo)' => 'foo_t'); # pass-by-pointer/reference
# new code:
use FFI::Platypus 1.00;
my $ffi = FFI::Platypus->new( api => 1 );
$ffi->type('record(Foo)' => 'foo_t'); # pass-by-value
$ffi->type('record(Foo)*' => 'foo_ptr'); # pass-by-pointer/reference
A minor tweak on the internals mean that you can also decorate aliases as pointer or array types.
# old code:
use FFI::Platypus;
my $ffi = FFI::Platypus->new;
$ffi->type('opaque' => 'o_t');
$ffi->type('opaque*' => 'o_ptr');
$ffi->type('opaque[10]' => 'o_array');
# XXX no records-by-value
$ffi->type('record(Foo)' => 'foo_ptr');
# new code:
use FFI::Platypus 1.00;
my $ffi = FFI::Platypus->new( api => 1 );
$ffi->type('opaque' => 'o_t');
$ffi->type('o_t*'); # pointer to o_t doesn't require an extra alias
$ffi->type('o_t[10]'); # same for arrays
$ffi->type('record(Foo)' => 'foo_t');
$fii->type('foo_t*'); # or pointers to records
Last, but not least, Platypus also now supports out of the box the common opaque pointer as an object pattern. TL;DR, given a C interface like this:
typedef struct foo_t;
foo_t* foo_new();
void foo_set_bar(foo_t*, const char *);
const char *foo_get_bar(foo_t*);
void foo_free(foo_t*);
You can write Perl like this:
# new code:
package Foo {
use FFI::Platypus 1.00;
my $ffi = FFI::Platypus->new( api => 1 );
$ffi->type('object(Foo)' => 'foo_t');
$ffi->mangler(sub {
my $name = shift;
$name =~ s/^foo_//r; # prefix all symbol lookups with foo_
});
$ffi->attach( new => [ ] => 'foo_t*' );
$ffi->attach( set_bar => [ 'foo_t*', 'string' ] );
$ffi->attach( get_bar => [ 'foo_t*' ] => 'string' );
$ffi->attach( free => [ 'foo_t*' ] );
sub DESTROY
{
my $self = shift;
$self->free;
}
}
my $foo = Foo->new;
$foo->set_bar("baz");
my $baz = $foo->get_bar;
If Platypus or the new API has piqued your interest or you have questions, we should definitely hang out! Join us on #native on irc.perl.org, check out the project on metacpan or github. If in person interaction is more your jam, then I will be at DCBPW in Baltimore next year along with TPCiH in Houston.
]]>First, lets suppose we have a very simple XS and Perl module that gives us the version of liblzma. This admittedly doesn't do anything very useful without the rest of the library, but it will help us test the basics of how to call a library that has been alienized. (if you need real LZMA support you should probably use IO::Compress::Lzma of course).
lib/LZMA/Example.pm (XS)
package LZMA::Example; use strict; use warnings; use base qw( Exporter ); our $VERSION = '0.01'; our \@EXPORT = qw( lzma_version_string ); require XSLoader; XSLoader::load('LZMA::Example', $VERSION); 1;
Example.xs
#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include "lzma.h" MODULE = LZMA::Example PACKAGE = LZMA::Example const char * lzma_version_string()
I will also write a very short test to make sure that everything is working (this test will be used for every single version that I am going to show today):
t/lzma_example.t
use Test2::Bundle::Extended; use LZMA::Example; my $version = lzma_version_string(); ok $version, 'returns a version'; note "version = $version"; done_testing;
Now Perl's philosophy is of course There's more than one way to do it, so we can use a variety of methods for building this as a Perl library. First we will show how to use ExtUtils::MakeMaker (EUMM), since it comes bundled with Perl. The challenge with EUMM is that setting the compiler and linker flags in the correct order can be tricky. Without a lot of effort it should work okay when you are using either the system or share install for Alien::xz, but if you forced a share install using the ALIEN_INSTALL_TYPE environment variable, then there is a very good chance that you will get the flag order wrong and use the wrong version of the library or to use the system library with your share install or vice versa. If that happens, you are going to have a bad day! To avoid this, I've written Alien::Base::Wrapper, which generates the correct flag order for you! Here is the example:
Makefile.PL
use strict; use warnings; use ExtUtils::MakeMaker; use Alien::Base::Wrapper qw( Alien::xz ); WriteMakefile( NAME => 'LZMA::Example', VERSION_FROM => 'lib/LZMA/Example.pm', CONFIGURE_REQUIRES => { 'ExtUtils::MakeMaker' => 6.52, 'Alien::xz' => '0.05', }, Alien::Base::Wrapper->mm_args, );
Although not used here, one of the neat things about Alien::Base::Wrapper is that you can use multiple Aliens in one Makefile.PL:
... use Alien::Base::Wrapper qw( Alien::Foo Alien::Bar Alien::Baz ); ...
But back to our LZMA::Example module. We can now build the module and run using perl and make.
% perl Makefile.PL Generating a Unix-style Makefile Writing Makefile for LZMA::Example Writing MYMETA.yml and MYMETA.json % make Skip blib/lib/LZMA/Example.pm (unchanged) Running Mkbootstrap for Example () chmod 644 "Example.bs" "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 clang -c -I/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/include -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" -fPIC "-I/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/x86_64-linux-thread-multi/CORE" Example.c rm -f blib/arch/auto/LZMA/Example/Example.so LD_RUN_PATH="/lib/x86_64-linux-gnu" clang -L/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/lib -pthread -shared -O2 -L/usr/local/lib -fstack-protector-strong Example.o -o blib/arch/auto/LZMA/Example/Example.so \ -llzma \ chmod 755 blib/arch/auto/LZMA/Example/Example.so % make test TEST_VERBOSE=1 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 PERL_DL_NONLAZY=1 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(1, 'blib/lib', 'blib/arch')" t/*.t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.3 1..1 ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.08 cusr 0.00 csys = 0.10 CPU) Result: PASS
We can do the same thing with Module::Build (MB), although that adds extra dependencies:
Build.PL
use strict; use warnings; use Module::Build; use Alien::xz; my $build = Module::Build->new( module_name => 'LZMA::Example', dist_abstract => 'lzma example', configure_requires => { 'Alien::xz' => '0.05', }, extra_compiler_flags => Alien::xz->cflags, extra_linker_flags => Alien::xz->libs, ); $build->create_build_script;
% perl Build.PL Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'LZMA-Example' version '0.01' % ./Build Building LZMA-Example clang -I/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/x86_64-linux-thread-multi/CORE -DVERSION="0.01" -DXS_VERSION="0.01" -fPIC -I/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/include -c -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -o lib/LZMA/Example.o lib/LZMA/Example.c ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/LZMA/Example/Example.bs') clang -shared -O2 -L/usr/local/lib -fstack-protector-strong -o blib/arch/auto/LZMA/Example/Example.so lib/LZMA/Example.o -L/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/lib -llzma % ./Build test verbose=1 t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.3 1..1 ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.01 usr 0.01 sys + 0.08 cusr 0.01 csys = 0.11 CPU) Result: PASS
Some people like Dist::Zilla instead:
dist.ini
name = LZMA-Example version = 0.01 abstract = LZMA example [@Filter] -bundle = @Basic -remove = MakeMaker [Prereqs / ConfigureRequires] Alien::xz = 0.05 [MakeMaker::Awesome] header = use Config; header = use Alien::xz; WriteMakefile_arg = CCFLAGS => Alien::xz->cflags . ' ' . $Config{ccflags} WriteMakefile_arg = LIBS => [ Alien::xz->libs ]
% dzil test [DZ] building distribution under .build/4IIoWEI6uU for installation [DZ] beginning to build LZMA-Example [DZ] writing LZMA-Example in .build/4IIoWEI6uU Checking if your kit is complete... Looks good Generating a Unix-style Makefile Writing Makefile for LZMA::Example Writing MYMETA.yml and MYMETA.json cp lib/LZMA/Example.pm blib/lib/LZMA/Example.pm Running Mkbootstrap for Example () chmod 644 "Example.bs" "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/ExtUtils/xsubpp" -typemap '/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/ExtUtils/typemap' Example.xs > Example.xsc Please specify prototyping behavior for Example.xs (see perlxs manual) mv Example.xsc Example.c clang -c -I/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/include -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" -fPIC "-I/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/x86_64-linux-thread-multi/CORE" Example.c rm -f blib/arch/auto/LZMA/Example/Example.so clang -shared -O2 -L/usr/local/lib -fstack-protector-strong Example.o -o blib/arch/auto/LZMA/Example/Example.so \ -L/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/lib -llzma \ chmod 755 blib/arch/auto/LZMA/Example/Example.so "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 PERL_DL_NONLAZY=1 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/lzma_example.t .. ok All tests successful. Files=1, Tests=1, 1 wallclock secs ( 0.02 usr 0.01 sys + 0.09 cusr 0.00 csys = 0.12 CPU) Result: PASS [DZ] all's well; removing .build/4IIoWEI6uU
As many may already know, an alternative way to write “XS” modules is by using Inline::C or Inline::CPP. One of the advantages to this is that the code can be compiled on demand as needed. Also all of your code can be contained within the Perl source file. Happily Alien::Base and all library modules that use it work quite nicely with Inline, and in order to integrate the two you can use the with keyword when writing your Inline module:
lib/LZMA/Example.pm (Inline)
package LZMA::Example; use strict; use warnings; use Inline with => 'Alien::xz'; use Inline C => <<'END'; #include <lzma.h> const char * _version_string() { return lzma_version_string(); } END use base qw( Exporter ); our $VERSION = '0.01'; our @EXPORT = qw( lzma_version_string ); sub lzma_version_string { _version_string(); } 1;
% prove -lvm t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.3 1..1 ok All tests successful. Files=1, Tests=1, 1 wallclock secs ( 0.03 usr 0.00 sys + 0.25 cusr 0.01 csys = 0.29 CPU) Result: PASS
One of the downsides to using Alien and Inline together like this is that the Alien module becomes a run-time dependency for your module. In the previous XS examples, Alien::xz was only a configure time dependency. This is likely mostly of concern to integrators building operating system packages for Perl modules, but it might be something to think about as a developer as well.
Another way to call a library from Perl is to use FFI. There are two usable FFI systems on CPAN, FFI::Raw and FFI::Platypus. (I personally recommend Platypus over Raw). FFI has some advantages and disadvantages over XS: your code is in one Perl file, it does not even need to be “built” or “installed” to be used, it introduces some additional complexity, since both Platypus and Raw are implemented using XS, it needs any Aliens as run-time dependencies (just as with Inline), etc.
Here is an example of our LZMA module using FFI::Platypus:
lib/LZMA/Example.pm (FFI)
package LZMA::Example; use strict; use warnings; use FFI::Platypus; use Alien::xz; use base qw( Exporter ); our $VERSION = '0.01'; our @EXPORT = qw( lzma_version_string ); my $ffi = FFI::Platypus->new( lib => [ Alien::xz->dynamic_libs ], ); $ffi->attach( lzma_version_string => [] => 'string' ); 1;
% prove -lvm t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.3 1..1 ok All tests successful. Files=1, Tests=1, 1 wallclock secs ( 0.03 usr 0.00 sys + 0.25 cusr 0.01 csys = 0.29 CPU) Result: PASS
Now I also mentioned that some Aliens can be used as tool oriented Alien modules. Alien::xz is an example of a hybrid Alien which can be used as a library oriented Alien (it provides liblzma) and as a tool oriented Alien (it provides the xz command line tool). Some Aliens, like Alien::gmake are tool oriented Aliens only, that is they provide a tool, but not library. Just to show how easy it is to use, lets write the same example module using the tool oriented mode:
lib/LZMA/Example.pm (Tool)
package LZMA::Example; use strict; use warnings; use Capture::Tiny qw( capture ); use Alien::xz; use Env qw( @PATH ); use base qw( Exporter ); our $VERSION = '0.01'; our @EXPORT = qw( lzma_version_string ); unshift @ENV, Alien::xz->bin_dir; sub lzma_version_string { my $out = capture { system 'xz', '--version' }; my($version) = $out =~ /liblzma ([0-9.]+)/; $version; } 1;
% prove -lvm t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.2 1..1 ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.13 cusr 0.00 csys = 0.15 CPU) Result: PASS
Alien::xz (and all modules that use Alien::Base) provides a bin_dir method, which returns the path to the tool if it is not already in the path, or empty list if the tool is already in the path. This means that once unshifted onto PATH, we can call it like any other command line tool. Some tools may have different names depending on the platform or the environment. Alien::gmake, for example, provides GNU Make, which may be called either gmake or make. It provides an exe method to tell you what the name is locally.
So, as you can see in the Perl TMTOWTDI spirit, there are many different ways to utilize an Alien module built using alienfile + Alien::Build + Alien::Base. There are some other systems out there like Module::Install (MI) that I didn't demonstrate. Because MI is not being actively developed any longer and EUMM is preferred for new development, there may be some out there using this older technology that could benefit from Alien. Or some other technology that I haven't thought of using with Alien yet. If you are one of those people please do feel free to stop on by the #native channel on irc.perl.org, or open an issue on the Alien::Build issue tracker. I have added the full working examples demonstrated here (with all the necessary support files) to the example/user directory in Alien::Build. I have also written a manual on using Alien::Base based Aliens here: Alien::Build::Manual::AlienUser. Next time I will demonstrate some specialized features, like how to use an Alien as a fallback prerequisite.
]]>The main motiviation for alienfile + Alien::Build was to separate the alien detection and installer code from the perl installer code. (In fact your alienfile is fully usable without any Perl installer at all; you can use your alienfile from a Perl script or Perl module using Alien::Build directly).
For our Alien, we will be creating Alien-xz, and we will use Alien::Build::MM to provide the thin layer of functionality needed between ExtUtils::MakeMaker (EUMM) and Alien::Build. This is what our Makefile.PL should look like:
use strict; use warnings; use ExtUtils::MakeMaker; use Alien::Build::MM; my %WriteMakefileArgs = ( "ABSTRACT" => "Find or build xz", "AUTHOR" => "Graham Ollis <plicease\@cpan.org>", "VERSION_FROM" => "lib/Alien/xz.pm", "CONFIGURE_REQUIRES" => { "ExtUtils::MakeMaker" => "6.52", }, "PREREQ_PM" => { "Alien::Base" => "0.038", }, "DISTNAME" => "Alien-xz", "LICENSE" => "perl", "NAME" => "Alien::xz", "PREREQ_PM" => { "Alien::Base" => "0.038" }, ); my $abmm = Alien::Build::MM->new; %WriteMakefileArgs = $abmm->mm_args(%WriteMakefileArgs); WriteMakefile(%WriteMakefileArgs); sub MY::postamble { $abmm->mm_postamble; }
Most of this will be pretty recognizable to anyone who has hand crafted a Makefile.PL before. After constructing the basic arguments that will be passed to WriteMakefile, we create an instance of Alien::Build::MM and pass the arguments into the mm_args method, which returns a modified version of those arguments. This method will decide if it can find xz and liblzma from what is provided by the operating system, or if it should build the tool and library from source. To make this decision it consults the alienfile. There are dynamic prerequisite that depend on the outcome of this decision. Typcially a build from source may pull in additional prerequisite to download or unpack an archive in an unusual format, or perhaps the build system requires an extra module or two. A system install may also require additional modules, although that is less common. At the bottom of the Makefile.PL we add a postamble to the Makefile. This will add the make targets needed to build xz.
Note that we declare Alien::Base as a prerequisite because our Alien class will be a subclass of that. We also declare EUMM as a configure time prerequisite which is a good practice for any EUMM dist. We do NOT need to declare Alien::Build::MM as a configure time prerequisite even though it is used in the Makefile.PL because that is one of the prerequisite that are addedin by mm_arg.
As mentioned earlier, alienfile is not tied to any one Perl installer, so if you prefer you can use Module::Build via Alien::Build::MB. I personally do not recommend this. It just adds extra prerequisite to your Alien. The sole reason I wrote Alien::Build::MB was to act as a proof of concept that: yes alienfile + Alien::Build can be used with any installer that meets its requirements. Including hopefully future installers that are more capable than EUMM. (Module::Build::Tiny will likely never be supported, as it is useful for a different subset of things).
If (like me) you do not like mucking about in Makefile.PL or Build.PL files you can instead use the Dist::Zilla plugin Dist::Zilla::Plugin::AlienBuild.
[MakeMaker] [AlienBuild] ; a little shorter than the Makefile.PL above eh?
The next thing that we need is the actual Perl module! We will call that lib/Alien/xz.pm, and will look like this:
package Alien::xz; use strict; use warnings; use base qw( Alien::Base ); our $VERSION = '1.00'; 1;
Not a lot is there? For most Aliens you will find that the base class does everything that you need. The only thing missing here really, (and I do not reproduce it here for the sake of brevity) is documentation. You should provide your users with enough information in the form of POD to be able to use this module! (See Alien::Build::Manual::AlienUser for clues as to what should be included). If you are lazy like me, you will want to use the Dist::Zilla plugin Dist::Zilla::Plugin::AlienBase::Doc to generate synopsis and description.
You should of course also provide other common distribution files, such as a MANIFEST and a Changes file, but all of that is beyond the scope of this document. (Always wanted to say that).
Now we can install our Alien like any other distribution. Create the make file:
% perl Makefile.PL
You can download the xz tarball using the alien_download target:
% make alien_download "/Users/ollisg/opt/perl/5.24.0/bin/perl" -MAlien::Build::MM=cmd -e prefix site /Users/ollisg/opt/perl/5.24.0/my/dev/lib/perl5/darwin-2level /Users/ollisg/opt/perl/5.24.0/my/dev/lib/perl5/darwin-2level /Users/ollisg/opt/perl/5.24.0/my/dev/lib/perl5/darwin-2level main> prefix /Users/ollisg/opt/perl/5.24.0/my/dev/lib/perl5/darwin-2level/auto/share/dist/Alien-xz "/Users/ollisg/opt/perl/5.24.0/bin/perl" -MAlien::Build::MM=cmd -e download Alien::Build::Plugin::Core::Download> decoding html Alien::Build::Plugin::Core::Download> candidate *http://tukaani.org/xz/xz-5.2.3.tar.gz Alien::Build::Plugin::Core::Download> setting version based on archive to 5.2.3 Alien::Build::Plugin::Core::Download> downloaded xz-5.2.3.tar.gz
(hint, if you are testing this and the system xz and liblzma are being detected, the download step will be a noisy NOOP. You can set ALIEN_INSTALL_TYPE to share to override this and force a source code build.)
You can then build xz and liblzma using the alien_build target:
% make alien_build "/Users/ollisg/opt/perl/5.24.0/bin/perl" -MAlien::Build::MM=cmd -e build Alien::Build::CommandSequence> + ./configure --prefix=/Users/ollisg/opt/perl/5.24.0/my/dev/lib/perl5/darwin-2level/auto/share/dist/Alien-xz --with-pic --disable-shared XZ Utils 5.2.3 System type: checking build system type... x86_64-apple-darwin15.6.0 checking host system type... x86_64-apple-darwin15.6.0 ...
(copious output not included).
You can also just do a regular make all and it will build the alien_download and alien_build targets, along with the necessary Perl specific targets.
Now we are ready to run the tests:
% make test PERL_DL_NONLAZY=1 "/Users/ollisg/opt/perl/5.24.0/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/run.t ...... ok t/xs.t ....... ok All tests successful. Files=2, Tests=7, 1 wallclock secs ( 0.03 usr 0.01 sys + 0.70 cusr 0.56 csys = 1.30 CPU) Result: PASS
I waited until now to remind you that you need tests! It is important to know that the Alien will work with your XS module. You don't want to find out it doesn't work when you are installing that. The best way to do this is to use Test::Alien, which tests your Alien with that same tools that your Alien will actually be used with. Here is a very basic Test::Alien test that ensures the alienized liblzma works correctly with XS:
use Test2::Bundle::Extended; use Test::Alien; use Alien::xz; alien_ok 'Alien::xz'; xs_ok do { local $/; <DATA> }, with_subtest { my $version = lzma::lzma_version_string(); ok $version; note "version = $version"; }; done_testing; __DATA__ #include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include <lzma.h> MODULE = lzma PACKAGE = lzma const char * lzma_version_string()
...and here is the tool test that ensures the alienized command line xz works correctly:
use Test2::Bundle::Extended; use Test::Alien; use Alien::xz; alien_ok 'Alien::xz'; run_ok(['xz', '--version']) ->success ->out_like(qr{XZ Utils}) ->note; done_testing;
You are all done writing your Alien. Although it may seem like you went through a lot, this is a lot less work than if you tried to roll your own Alien. Now we can finally install our Alien, and just eyeball test that it works from the command line.
% make install ... % perl -MAlien::xz -E 'say Alien::xz->version' 5.2.3 % perl -MAlien::xz -E 'say Alien::xz->cflags' -I/Users/ollisg/opt/perl/5.24.0/my/dev/lib/perl5/darwin-2level/auto/share/dist/Alien-xz/include % perl -MAlien::xz -E 'say Alien::xz->libs' -L/Users/ollisg/opt/perl/5.24.0/my/dev/lib/perl5/darwin-2level/auto/share/dist/Alien-xz/lib -llzma
Next time we will use the Alien that we have crafted here from an XS or FFI module, which is ultimately the reason for all of this prep work.
Correction: a previous version of this blogity blog incorrectly referred to Alien::Build::MM as a dynamic prerequisite. It is always added so it is strictly speaking a static prerequisite.
]]>So when I was working on:
intended as a modern pluggable alternative to ABMB, I wanted to make sure this new tech did these things at least as well. (In a github issue I mentioned the motivations for such a move, but briefly, ABMB is tightly coupled with Module::Build which has fallen out of favor and is no longer distributed as part of the Perl Core). In this dispatch I will cover how alienfile + Alien::Build make these things easy. In dispatches further down the line I will demonstrate what you can do with alienfile which is either hard or impossible with ABMB.
Most software package provide instructions on how to install them as a list of commands that you type into your shell. For example, the xz package which provides the command line program xz and the library liblzma can be installed using the standard procedure for autoconf based packages:
% ./configure % make % make install
So intuitively the simplest way to start with alienfile is to put those commands into an alienfile like this:
use alienfile; # This example is borrowed from the examples directory of the # Alien-Build distribution. # Use pkg-config to check if the library exists. # also, use which to check that the xz command is # in the path. probe [ 'pkg-config --exists liblzma', 'which xz', ]; # If the probe fails to find an already installed xz, the # "share" block contains instructions for downloading and # installing it. share { # the first download which succeeds will be used download [ 'wget http://tukaani.org/xz/xz-5.2.3.tar.gz' ]; download [ 'curl -O http://tukaani.org/xz/xz-5.2.3.tar.gz' ]; # use tar to extract the tarball extract [ 'tar zxf %{.install.download}' ]; # use the standard build process build [ './configure --prefix=%{.install.prefix} --disable-shared', '%{make}', '%{make} install', ]; }; # gather the details necessary for using the library, and store them # as runtime properties. gather [ # store the (chomped) output into the appropriate runtime properties [ 'pkg-config', '--modversion', 'liblzma', \'%{.runtime.version}' ], [ 'pkg-config', '--cflags', 'liblzma', \'%{.runtime.cflags}' ], [ 'pkg-config', '--libs', 'liblzma', \'%{.runtime.libs}' ], ];
If you have App::af installed, you can test this alienfile with the af download command:
% af download -f alienfile Alien::Build::CommandSequence> + wget http://tukaani.org/xz/xz-5.2.3.tar.gz --2017-03-25 12:48:14-- http://tukaani.org/xz/xz-5.2.3.tar.gz Resolving tukaani.org... 84.34.147.45 Connecting to tukaani.org|84.34.147.45|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1490665 (1.4M) [application/x-gzip] Saving to: 'xz-5.2.3.tar.gz' xz-5.2.3.tar.gz 100%[=========================================================>] 1.42M 482KB/s in 3.0s 2017-03-25 12:48:17 (482 KB/s) - 'xz-5.2.3.tar.gz' saved [1490665/1490665] Alien::Build> single file, assuming archive Wrote archive to /Users/ollisg/dev/Alien-Build/example/xz-5.2.3.tar.gz
...and the af install command:
% af install -f alienfile --prefix=/tmp/foo ... lots of output... copied staged install into /tmp/foo --- cflags: -I/tmp/foo/include cflags_static: -I/tmp/foo/include install_type: share legacy: finished_installing: 1 install_type: share original_prefix: /tmp/foo version: 5.2.3 libs: -L/tmp/foo/lib -llzma libs_static: -L/tmp/foo/lib -llzma prefix: /tmp/foo version: 5.2.3
As you can see an alienfile is a series of steps which can be specified as a list of commands. Commands can use a simple macro or helper facility for portability. For example, instead of using make directly, it is advisable to use %{make}, since it will be replaced by dmake or nmake on platforms that use those names instead. Some packages require GNU Make, in which case you can use %{gmake} and be sure you get the right type of make (it is a good idea to request the upstream developer to add support for other versions of make for better portability too). If a command fails then it will stop the install. Steps can also be specified as a code reference when they can more easily be expressed as Perl code. If the code reference throws an exception it will stop the install. These are the basic steps that an alienfile will need to define in most cases:
There is also a sys block for steps that happen only during an install when the library or tool is already installed on the system, and a gather step, which gathers the details on how to use the library or tool that will be needed by the Alien runtime when the Alien is installed. The gather step can be placed in either or both of the share or sys block, allowing different gather mechanisms to be used depending on the install type. You can also put the gather (as in the example above) step outside of either bock, which is the same as putting the identical instructions in both blocks.
This manual example is great for seeing how alienfile works, but typically for autoconf and pkg-config based projects you will want to use plugins:
use alienfile; plugin 'PkgConfig' => 'liblzma'; plugin 'Probe::CommandLine' => ( command => 'xz', secondary => 1, ); share { plugin Download => ( url => 'http://tukaani.org/xz/', version => qr/^xz-([0-9\.]+)\.tar\.gz$/, ); plugin Extract => 'tar.gz'; plugin 'Build::Autoconf' => (); # the build step is only necessary if you need to customize the # options passed to ./configure. The default set by the # Build::Autoconf plugin is frequently sufficient. build [ '%{configure} --disable-shared', '%{make}', '%{make} install', ]; };
This does more or less the same thing as the previous example with a much more concise recipe. It does a number of things better than the first example though:
So that is a lot already. I can tell as your eyes are starting to glaze over! And you have already learned a lot. Next time I will show you how to integrate your alienfile recipe into an Alien distribution and how to use it from your XS module.
]]>A year ago I mentioned that we were planning on spinning Alien::Base::ModuleBuild (AB::MB) off from the main Alien::Base distribution. Now that we have a viable alternative, we plan to split AB::MB off into its own distribution next week. Alien::Base will specify AB::MB as a prerequisite, until the first of October 2017. At that time it will be removed as a prerequisite and if you are using AB::MB you will need to specify it as a configure_requires in your Build.PL.
The main potential breakage for this is that when trying to install an Alien which hasn't been
fixed, you will receive an error message like this:
Can't locate Alien/Base/ModuleBuild.pm in @INC
I think most Perl programmers will know that they need to install AB::MB and this is a very minor inconvenience. In addition, because we have been aggressive about notifying Alien developers and providing pull requests there are very few Alien modules that still have this bug. Those that do are not used widely .
We plan on maintaining AB::MB for quite some time, as there is a lot of working code that uses it, and works well.
]]>This is technically more correct, and it will also future proof your module in the event that Alien::Base::ModuleBuild gets spun off from the rest of Alien::Base. There are a number of motivations making such a move. Please join us on GitHub or the #native IRC channel if are interested in working on the next generation of of Alien::Base.
]]>The one major platform that didn’t work on the initial switch was of course Strawberry Perl, but after some debugging and patches I got Alien::Hunspell and Text::Hunspell to work there as well. I even submitted patches to upstream to the hunspell project, which were accepted, so that in the future less patching will be required. This is what is great about Open Source when it works.
The results as recorded in the cpantesters matrix are stark:
In the process of working on all of this I finally gave up on Test::CChecker. I wrote this module to test Alien distributions, and leveraged some capabilities from ExtUtils::CChecker, which is designed to probe and check compiler and linker flags from a Makefile.PL or Build.PL file.
Test::CChecker was a huge improvement over existing state of affairs for testing Aliens when I wrote it. For most such modules, this consisted of the hand rolled .t files that used the Config module and attempted to build executables by invoking the compiler via system or qx. Assuming that an Alien module even had any tests. Here is a simplified version of the Test::CChecker test that I wrote for Alien::Hunspell.
use Test::More; use Alien::Hunspell; use Test::CChecker 0.07;plan tests => 2;
compile_output_to_note;
compile_with_alien 'Alien::Hunspell';
my $source = do { local $/; <DATA> };
compile_ok $source, 'basic compile test';
compile_run_ok $source, "basic compile/link/run test";
__DATA__
#include <hunspell.h>int
main(int argc, char *argv[])
{
Hunhandle *h;
h = Hunspell_create("","");
Hunspell_destroy(h);
return 0;
}
What this does is compile and link an executable using the flags provided by Alien::Hunspell. It then runs it and ensures that it doesn’t dump core, and returns a successful exit status. Simple(ish) for Alien right? The problem is that there are a number of subtle edge cases that become much more prominent when you are installing an Alien module that decides that it needs to build the upstream package from source. The biggest difference between what this test tests, and how the Alien module is actually used is that Text::Hunspell creates a dynamic library and links it to the running Perl process. This means that the test file catches errors specific to linking executables that aren’t pertinent to usage in Perl. It also means that this test file does NOT catch errors that are specific to dynamic libraries, which ARE pertinent to usage in Perl.
So I wrote Test::Alien to test Alien modules in the way that they are actually used. Create an interface to make it easy to create a mini-XS or FFI and run some basic tests like ensure you can query the version number. This increases the likelihood that the Alien module will actually be useful dramatically, and it catches errors with the Alien module where they should be found, in the Alien module itself, not in the downstream XS or FFI module. Here are simplified tests, also for Alien::Hunspell:
# test for XS use Test::Stream -V1; use Test::Alien; use Alien::Hunspell;plan 3;
alien_ok 'Alien::Hunspell';
my $xs = do { local $/; <DATA> };
xs_ok { xs => $xs, verbose => 1 }, with_subtest {
my $ptr = My::Hunspell::Hunspell_create("t/supp.aff","t/supp.dic");
ok $ptr, "ptr = $ptr";
My::Hunspell::Hunspell_destroy($ptr);
ok 1, "did not crash";
};__DATA__
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include <hunspell/hunspell.h>MODULE = My::Hunspell PACKAGE = My::Hunspell
void *
Hunspell_create(affpath, dpath);
const char *affpath;
const char *dpath;void
Hunspell_destroy(handle);
void *handle;
This tests that a basic dynamic extension can be built, and can create and destroy a simple hunspell instance without crashing.
# test for FFI use Test::Stream -V1; use Test::Alien; use Alien::Hunspell;plan 3;
alien_ok 'Alien::Hunspell';
ffi_ok { symbols => [qw( Hunspell_create Hunspell_destroy )] }, with_subtest {
my($ffi) = @_;
plan 2;
$ffi->attach(Hunspell_create => ['string','string'] => 'opaque');
my $ptr = Hunspell_create("t/supp.aff", "t/supp.dic");
ok $ptr, "ptr = $ptr";
$ffi->attach(Hunspell_destroy => ['opaque'] => 'void');
Hunspell_destroy($ptr);
ok 1, "did not crash";
};
Test::Alien also has tests for tool oriented Alien modules, such as Alien::gmake and Alien::patch. Test::Alien is designed to work seamlessly with Alien::Base based Alien modules, but can also be made to work without much additional work with non Alien::Base based Aliens.
The other nice thing about the Test::Alien interface is that its tests will self skip if a compiler or FFI::Platypus are not found. This is useful, as Text::Hunspell::FFI can use Alien::Hunspell without a compiler if the hunspell system packages are installed, and conversely Text::Hunspell can use Alien::Hunspell if FFI::Platypus is not installed.
I used Test::Alien in development versions of Alien::Hunspell to identify and correct bugs to make it more reliable.
]]>Use %c instead of %pconfigure.
Long story short, the %p directive was intended to handle the portability problem when running a command in the current directly. On Unix this requires a ./ prefix. That won’t work on Windows. Unfortunately, configure is usually a shell script and on Windows you need MSYS (or something similar) to provide sh and friends. Even if you had MSYS, %pconfigure won’t correctly invoke configure on Windows, because that isn’t how you run a shell script on Windows. So back in version 0.005 we added the %c directive to mean “run configure however that works on this platform”. If AB sees that you are using %c it will also make sure that Alien::MSYS gets added as a build requirement, if it is needed. In many cases, adding Windows support for your AB based Alien distribution may be as simple replacing %pconfigure with %c and making AB 0.005 a prerequisite.
Require Alien::Base 0.018 (or 0.021)
In version 0.016 we introduced staged installs to blib, and in 0.018 we made it the default. This was a significant change so while believed it shouldn't break any existing modules, out an abundance of caution, I blogged about this a while ago, and included some of the technical details:
So far, this change has been pretty positive. The upshot is that the cpan tester installs work in the same way as regular installs that users make. This means that the results in your cpan testers matrix is more reliable AND more representative as to how users are actually using your module. I strongly recommend that anyone using AB bump the required version up to 0.018. If your package doesn’t use pkg-config consider requiring 0.021 which fixed a bug with system libraries that don’t have a pkg-config .pc file.
edit: The original author has uploaded a new version with these (and a number of other) issues resolved!
]]>