tinita

On thursday evening and friday of the Hackweek I decided to work on the integration of YAML::PP and libyaml.

Previous hackweek posts:

• Day 1 - Perl ♥ Bash
• Day 2 - YAML::PP !include
• Day 3 - (Not) Loading Objects in YAML::PP

Enhancing the YAML::LibYAML::API module

Today, the YAML::LibYAML::API module is a very simple wrapper around the C libyaml.

You create an empty array reference and then pass that to the parse_string_events (or parse_file_events, parse_filehandle_events) function.

my $events = [];
YAML::LibYAML::API::XS::parse_string_events($yaml, $events);

When parsing is done, you can then do stuff with the events.

This is how YAML::PP::LibYAML is using it currently.

This has some disadvantages, although they are not very relevant for simple usage.

My plan was to create a perl object that has a "reference" to the C struct yaml_parser_t, so I can keep it around and refer to it later.

This will make it possible in the future to query information from the parser, for example getting the exact error details in case of a parsing error.

Another possibility comes to mind when handling large YAML files with many documents, and you only want to load one document.

Fighting with XS and C

So I had to learn how to keep a C struct around. The solution in the end was pretty simple, but I was trying many things, and fortunately Harald Jörg and others helped me very patiently when I asked questions about this on IRC. Thanks a lot!

For keeping the parser struct around, you have to malloc memory for it:

yaml_parser_t *parser;
parser = malloc(sizeof(yaml_parser_t));

Now, I would like to "reference" this struct in my perl object. I was trying to return an integer to XS, and somehow I wasn't able to get the struct back. It turned out I had to use a uintptr_t and long:

return (long) (uintptr_t) parser;

In the XS code it's still sufficient to return an SViv:

RETVAL = newSViv(id);

My perl object looks like that:

bless { uid => 1234567 }, 'YAML::LibYAML::API::XS'

To access the parser struct again in XS/C, the following is necessary:

    HV *hash;
    SV* obj_sv;
    SV **sv;
    long id;
    yaml_parser_t *parser;

    SvGETMAGIC(obj);
    if (!SvROK(obj))
        croak("Not a reference");

    obj_sv = SvRV(obj);
    if (SvTYPE(obj_sv) != SVt_PVHV)
        croak("Not a reference to a hash");
    hash = (HV *)(obj_sv);

    sv = hv_fetch(hash, "uid", 3, TRUE);
    if (!sv) {
        croak("%s\n", "Could not get uid");
    }
    id = (long) SvIV(*sv);

    parser = (yaml_parser_t*) (uintptr_t) id;

Adding a callback for parsing

Now I am able to let C call a perl callback function whenever it gets a new parsing event. This is also how the YAML::PP::Parser backend is working, so the integration will be a bit nicer.

This is how it currently looks:

my $xsparser = YAML::LibYAML::API::XS->new;
$xsparser->parser_create;

my $yaml = <<'EOM';
---
a: b
EOM

$xsparser->parser_init_string($yaml);
my $cb = sub {
    my ($event) = @_;
    warn Dumper $event;
};

$xsparser->set_parse_callback($cb);
$xsparser->parse_callback();
$xsparser->parser_delete();

Everything is still in a branch. I have to add exception handling and refactor a bit.

And, of course I have to add the same kind of thing to the Emitter.

End of the Hackweek

Many thanks to SUSE again for dedicating a week of everyone's worktime for stuff like this!

I think every company using Open Source should do this once in a while.