The many ways to use Alien
A while back I introduced the alienfile recipe system and we wrote a simple alienfile that provides in a CPAN context the tool xz and the library liblzma. I also went over how to test it with App::af. The week after that I showed how to integrate that alienfile into a fully functioning Alien called Alien::xz and promised to show how to then use that Alien from an XS or FFI module. Today I am going to do that. I am also going to show how to use a tool oriented Alien module. (conveniently, Alien::xz can be used in either library or tool oriented Alien mode). If you are more interested in FFI or tool oriented mode feel free to skip down to the appropriate paragraph.
First, lets suppose we have a very simple XS and Perl module that gives us the version of liblzma. This admittedly doesn't do anything very useful without the rest of the library, but it will help us test the basics of how to call a library that has been alienized. (if you need real LZMA support you should probably use IO::Compress::Lzma of course).
lib/LZMA/Example.pm (XS)
package LZMA::Example;
use strict;
use warnings;
use base qw( Exporter );
our $VERSION = '0.01';
our \@EXPORT = qw( lzma_version_string );
require XSLoader;
XSLoader::load('LZMA::Example', $VERSION);
1;
Example.xs
#include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include "lzma.h" MODULE = LZMA::Example PACKAGE = LZMA::Example const char * lzma_version_string()
I will also write a very short test to make sure that everything is working (this test will be used for every single version that I am going to show today):
t/lzma_example.t
use Test2::Bundle::Extended; use LZMA::Example; my $version = lzma_version_string(); ok $version, 'returns a version'; note "version = $version"; done_testing;
Now Perl's philosophy is of course There's more than one way to do it, so we can use a variety of methods for building this as a Perl library. First we will show how to use ExtUtils::MakeMaker (EUMM), since it comes bundled with Perl. The challenge with EUMM is that setting the compiler and linker flags in the correct order can be tricky. Without a lot of effort it should work okay when you are using either the system or share install for Alien::xz, but if you forced a share install using the ALIEN_INSTALL_TYPE environment variable, then there is a very good chance that you will get the flag order wrong and use the wrong version of the library or to use the system library with your share install or vice versa. If that happens, you are going to have a bad day! To avoid this, I've written Alien::Base::Wrapper, which generates the correct flag order for you! Here is the example:
Makefile.PL
use strict;
use warnings;
use ExtUtils::MakeMaker;
use Alien::Base::Wrapper qw( Alien::xz );
WriteMakefile(
NAME => 'LZMA::Example',
VERSION_FROM => 'lib/LZMA/Example.pm',
CONFIGURE_REQUIRES => {
'ExtUtils::MakeMaker' => 6.52,
'Alien::xz' => '0.05',
},
Alien::Base::Wrapper->mm_args,
);
Although not used here, one of the neat things about Alien::Base::Wrapper is that you can use multiple Aliens in one Makefile.PL:
... use Alien::Base::Wrapper qw( Alien::Foo Alien::Bar Alien::Baz ); ...
But back to our LZMA::Example module. We can now build the module and run using perl and make.
% perl Makefile.PL Generating a Unix-style Makefile Writing Makefile for LZMA::Example Writing MYMETA.yml and MYMETA.json % make Skip blib/lib/LZMA/Example.pm (unchanged) Running Mkbootstrap for Example () chmod 644 "Example.bs" "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 clang -c -I/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/include -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" -fPIC "-I/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/x86_64-linux-thread-multi/CORE" Example.c rm -f blib/arch/auto/LZMA/Example/Example.so LD_RUN_PATH="/lib/x86_64-linux-gnu" clang -L/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/lib -pthread -shared -O2 -L/usr/local/lib -fstack-protector-strong Example.o -o blib/arch/auto/LZMA/Example/Example.so \ -llzma \ chmod 755 blib/arch/auto/LZMA/Example/Example.so % make test TEST_VERBOSE=1 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 PERL_DL_NONLAZY=1 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(1, 'blib/lib', 'blib/arch')" t/*.t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.3 1..1 ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.08 cusr 0.00 csys = 0.10 CPU) Result: PASS
We can do the same thing with Module::Build (MB), although that adds extra dependencies:
Build.PL
use strict;
use warnings;
use Module::Build;
use Alien::xz;
my $build = Module::Build->new(
module_name => 'LZMA::Example',
dist_abstract => 'lzma example',
configure_requires => {
'Alien::xz' => '0.05',
},
extra_compiler_flags => Alien::xz->cflags,
extra_linker_flags => Alien::xz->libs,
);
$build->create_build_script;
% perl Build.PL
Created MYMETA.yml and MYMETA.json
Creating new 'Build' script for 'LZMA-Example' version '0.01'
% ./Build
Building LZMA-Example
clang -I/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/x86_64-linux-thread-multi/CORE -DVERSION="0.01" -DXS_VERSION="0.01" -fPIC -I/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/include -c -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -o lib/LZMA/Example.o lib/LZMA/Example.c
ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/LZMA/Example/Example.bs')
clang -shared -O2 -L/usr/local/lib -fstack-protector-strong -o blib/arch/auto/LZMA/Example/Example.so lib/LZMA/Example.o -L/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/lib -llzma
% ./Build test verbose=1
t/lzma_example.t ..
# Seeded srand with seed '20170611' from local date.
ok 1 - returns a version
# version = 5.2.3
1..1
ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.01 usr 0.01 sys + 0.08 cusr 0.01 csys = 0.11 CPU)
Result: PASS
Some people like Dist::Zilla instead:
dist.ini
name = LZMA-Example
version = 0.01
abstract = LZMA example
[@Filter]
-bundle = @Basic
-remove = MakeMaker
[Prereqs / ConfigureRequires]
Alien::xz = 0.05
[MakeMaker::Awesome]
header = use Config;
header = use Alien::xz;
WriteMakefile_arg = CCFLAGS => Alien::xz->cflags . ' ' . $Config{ccflags}
WriteMakefile_arg = LIBS => [ Alien::xz->libs ]
% dzil test [DZ] building distribution under .build/4IIoWEI6uU for installation [DZ] beginning to build LZMA-Example [DZ] writing LZMA-Example in .build/4IIoWEI6uU Checking if your kit is complete... Looks good Generating a Unix-style Makefile Writing Makefile for LZMA::Example Writing MYMETA.yml and MYMETA.json cp lib/LZMA/Example.pm blib/lib/LZMA/Example.pm Running Mkbootstrap for Example () chmod 644 "Example.bs" "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/ExtUtils/xsubpp" -typemap '/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/ExtUtils/typemap' Example.xs > Example.xsc Please specify prototyping behavior for Example.xs (see perlxs manual) mv Example.xsc Example.c clang -c -I/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/include -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" -fPIC "-I/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/lib/5.26.0/x86_64-linux-thread-multi/CORE" Example.c rm -f blib/arch/auto/LZMA/Example/Example.so clang -shared -O2 -L/usr/local/lib -fstack-protector-strong Example.o -o blib/arch/auto/LZMA/Example/Example.so \ -L/home/ollisg/.perlbrew/libs/perl-5.26.0tc@dev/lib/perl5/x86_64-linux-thread-multi/auto/share/dist/Alien-xz/lib -llzma \ chmod 755 blib/arch/auto/LZMA/Example/Example.so "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" -MExtUtils::Command::MM -e 'cp_nonempty' -- Example.bs blib/arch/auto/LZMA/Example/Example.bs 644 PERL_DL_NONLAZY=1 "/home/ollisg/perl5/perlbrew/perls/perl-5.26.0tc/bin/perl5.26.0" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/lzma_example.t .. ok All tests successful. Files=1, Tests=1, 1 wallclock secs ( 0.02 usr 0.01 sys + 0.09 cusr 0.00 csys = 0.12 CPU) Result: PASS [DZ] all's well; removing .build/4IIoWEI6uU
As many may already know, an alternative way to write “XS” modules is by using Inline::C or Inline::CPP. One of the advantages to this is that the code can be compiled on demand as needed. Also all of your code can be contained within the Perl source file. Happily Alien::Base and all library modules that use it work quite nicely with Inline, and in order to integrate the two you can use the with keyword when writing your Inline module:
lib/LZMA/Example.pm (Inline)
package LZMA::Example;
use strict;
use warnings;
use Inline with => 'Alien::xz';
use Inline C => <<'END';
#include <lzma.h>
const char * _version_string()
{
return lzma_version_string();
}
END
use base qw( Exporter );
our $VERSION = '0.01';
our @EXPORT = qw( lzma_version_string );
sub lzma_version_string
{
_version_string();
}
1;
% prove -lvm t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.3 1..1 ok All tests successful. Files=1, Tests=1, 1 wallclock secs ( 0.03 usr 0.00 sys + 0.25 cusr 0.01 csys = 0.29 CPU) Result: PASS
One of the downsides to using Alien and Inline together like this is that the Alien module becomes a run-time dependency for your module. In the previous XS examples, Alien::xz was only a configure time dependency. This is likely mostly of concern to integrators building operating system packages for Perl modules, but it might be something to think about as a developer as well.
Another way to call a library from Perl is to use FFI. There are two usable FFI systems on CPAN, FFI::Raw and FFI::Platypus. (I personally recommend Platypus over Raw). FFI has some advantages and disadvantages over XS: your code is in one Perl file, it does not even need to be “built” or “installed” to be used, it introduces some additional complexity, since both Platypus and Raw are implemented using XS, it needs any Aliens as run-time dependencies (just as with Inline), etc.
Here is an example of our LZMA module using FFI::Platypus:
lib/LZMA/Example.pm (FFI)
package LZMA::Example; use strict; use warnings; use FFI::Platypus; use Alien::xz; use base qw( Exporter ); our $VERSION = '0.01'; our @EXPORT = qw( lzma_version_string ); my $ffi = FFI::Platypus->new( lib => [ Alien::xz->dynamic_libs ], ); $ffi->attach( lzma_version_string => [] => 'string' ); 1;
% prove -lvm t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.3 1..1 ok All tests successful. Files=1, Tests=1, 1 wallclock secs ( 0.03 usr 0.00 sys + 0.25 cusr 0.01 csys = 0.29 CPU) Result: PASS
Now I also mentioned that some Aliens can be used as tool oriented Alien modules. Alien::xz is an example of a hybrid Alien which can be used as a library oriented Alien (it provides liblzma) and as a tool oriented Alien (it provides the xz command line tool). Some Aliens, like Alien::gmake are tool oriented Aliens only, that is they provide a tool, but not library. Just to show how easy it is to use, lets write the same example module using the tool oriented mode:
lib/LZMA/Example.pm (Tool)
package LZMA::Example;
use strict;
use warnings;
use Capture::Tiny qw( capture );
use Alien::xz;
use Env qw( @PATH );
use base qw( Exporter );
our $VERSION = '0.01';
our @EXPORT = qw( lzma_version_string );
unshift @ENV, Alien::xz->bin_dir;
sub lzma_version_string
{
my $out = capture { system 'xz', '--version' };
my($version) = $out =~ /liblzma ([0-9.]+)/;
$version;
}
1;
% prove -lvm t t/lzma_example.t .. # Seeded srand with seed '20170611' from local date. ok 1 - returns a version # version = 5.2.2 1..1 ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.13 cusr 0.00 csys = 0.15 CPU) Result: PASS
Alien::xz (and all modules that use Alien::Base) provides a bin_dir method, which returns the path to the tool if it is not already in the path, or empty list if the tool is already in the path. This means that once unshifted onto PATH, we can call it like any other command line tool. Some tools may have different names depending on the platform or the environment. Alien::gmake, for example, provides GNU Make, which may be called either gmake or make. It provides an exe method to tell you what the name is locally.
So, as you can see in the Perl TMTOWTDI spirit, there are many different ways to utilize an Alien module built using alienfile + Alien::Build + Alien::Base. There are some other systems out there like Module::Install (MI) that I didn't demonstrate. Because MI is not being actively developed any longer and EUMM is preferred for new development, there may be some out there using this older technology that could benefit from Alien. Or some other technology that I haven't thought of using with Alien yet. If you are one of those people please do feel free to stop on by the #native channel on irc.perl.org, or open an issue on the Alien::Build issue tracker. I have added the full working examples demonstrated here (with all the necessary support files) to the example/user directory in Alien::Build. I have also written a manual on using Alien::Base based Aliens here: Alien::Build::Manual::AlienUser. Next time I will demonstrate some specialized features, like how to use an Alien as a fallback prerequisite.
Perl programmer with an interest in Alien and FFI technologies. Primary developer of FFI::Platypus and Alien::Build.
Leave a comment