XS, Enough Said
Well I think I will tackle a difficult subject for my next series and that is XS programming or how to add C to you Perl and learn to regret it.
XS programming is not for the faint of heart I will warn you know it is tedious, stressful, unrewarding, undocumented and hard to learn. However if you can get past the first few high hurdles it is well worth the effort.
Now XS programming is not rare you can find it all over CPAN and I can almost grantee if you have ever used Perl to log into a DB then you have used XS.
So it is there and in use every day, it just works and we are happy about that.
So lets start at the beginning for me early one day after, about two months into a new job, and just getting back to Perl after an absence of about a dozen years. The boss called me in to his office where he had Tim Bunce on the line and it was decided that I was going to start maintaining DBD::Oracle.
Ok I said then I had a look at the code
- 5300+ lines of C code
- 1000+ lines of XS code
- 800+ lines of Perl and a
- 1800+ line Makefile.PL
Well lets just say I was a little taken aback but I stuck with it and over about 5+ year of being the prime maintainer I got very good at it.
Well at the time I did a number of Google searches and
- What there was, wasn't much
- What there was, didn't help much
- What there was usually ended with 'Read Perl Guts'
So it was the school of bitter experience for me, looking at code reading endless tutorials that just did 'Hello World'. I did most of my learning by just reading other CPAN modules that used XS and sometimes I got lucky other times not. Once I was trying to figure out how to return a hash from XS back to Perl. I spend 2 days going over every DBD Mod line by line until I finally found what I was looking for.
I was hoping these days there would be a little more since 2005 I was wrong there is still only one Book on the subject, now out of print, there are one or two new tutorials but now I just have 13 ways to do 'Hello World' instead of the 10 I had 5 years ago.
Oh well, so what to do? Well I wold like to do add something useful here to the XS world not just another 'hello world' or alike. What I do have is a complicated patch for DBD::Oracle sitting around someplace.
I did this for a client about 4 years ago and just never bothered to stick it in DBD::Oracle, well the real reason was I loss access to a box that has the correct enterprise version and the correct environment setup but I now have one of those handy so there is no excuse anymore so that will be my subject for this series.
So to leave you with a little for tonight here is a Perl Monk writeup on the subject that sort of sums up allot about XS but again like most XS postings it is over 4 years old.
Long time Perl guy, a few CPAN mods allot of work on DBD::Oracle and a few YAPC presentations
There might not be many great XS tutorials, but there's an #xs channel on the Perl IRC server, and I've found it a very helpful resource.
I'm sorry to hear that book has gone out of print, because it was the best resource out there.
Your description of XS is pretty good; murky, hard to figure out, and not clearly explained.
It's all C preprocessor macros; figure out what they do, and then figure out the interfaces, and it becomes clearer, but it's a big learning curve.
I have the book. It is helpful as you say, but you're right that it's still a steep learning curve, especially to figure out why your project is segfaulting inside the perl interpreter.
It helps to build and use a perl interpreter built with debugging.
I also use Inline:: to make my life much easier. Usually, I can get away with Inline, C, and some typemaps.
Because the connection between perl and C land can be so tedious, you have a very very strong incentive to think hard about getting the interface to a minimum. For me, this usually means the compiled side having a bunch of perl/accessible setter/getters, and some "action" functions, and having a perl wrapper in perl that makes that look/feel a bit more perlish. Maybe not optimal, but easy to code.