tinita

Perl Toolchain Summit Lyon, France, May 11-14 2017

This year I was invited to the Perl Toolchain Summit in Lyon, formerly known as the Perl QA Hackathon. I had organized this event two years ago in Berlin.

Having been "only" an organizer before, this time I could actually code. It was a pleasure to work with over 35 people doing a lot of work for the Perl 5 and 6 infrastructure and related topics.

One of them was Ingy döt Net, who I enjoy working with on Perl and other Open Source projects regularly remotely. Since we both love the commandline, I could learn a lot from Ingy over the last two years.

At the summit we were working on existing and new YAML modules. This is Ingy's report of what we did. Spoiler: We killed YAML.pm and reinvented it.

YAML Summit Berlin, Germany, May 5-8 2017

In this post I would like to talk about the 2017 YAML Summit that took place just one week before the Toolchain Summit.

Ingy döt Net and Felix Krause have been initiating/working on some new YAML projects throughout the last year, and at some point I joined them. Felix wrote a YAML parser in the Nim language. The wish for a new version of the YAML Spec came up. Since Ingy was coming to Europe for the Toolchain Summit, he had the idea to meetup in Berlin, and luckily Felix was able to come from Stuttgart very spontaneously.

I invited them to come to my place, and so we held the first ever YAML Summit!

We focused completely on the future versions: YAML 1.3 and YAML 2.0.

The goals for 1.3 were:

• Make YAML easier to read and parse
• Stay 99.9% backwards compatible for the YAML we know to be used in the wild
• Remove many little known spec anomalies that nobody uses, but that make YAML much harder to implement

The goals for 2.0 were:

• Things we'd like to see a future YAML become
• Break some backwards compatibility
• Can wait for a longer time

Our discussions resulted in 15 RFCs (so far), mostly for 1.3: RFC Index

We talked about merge keys, which we all agreed needed to be removed. Of course we have to come up with a better replacement. Also, mixing YAML with a templating language, like Ansible does with Jinja, is an issue. It would be cool if YAML could provide such a language on a syntax level instead of depending on the content, like merge keys and Jinja syntax.

Felix got an idea and wrote an RFC for something Ingy later called "Annotations".

YAML Projects

To give you an idea on what we've been working on until that Summit, here are three projects:

YAML Test Suite

Ingy and Felix created a format for YAML parser events that can be used to specify tests. As of this writing it contains 193 test files, each of them containing a description, the input YAML, the list of parser events, an output YAML, and, if possible, a JSON input. The tests are written in the TestML format. Additionally there is a data branch which contains all the data for each test in seperate files.

YAML Editor

Sometimes you want to know how different frameworks in different languages are parsing your YAML files. It's a lot of work to get them all running, so Ingy started the YAML editor, which builds a docker image with a number of frameworks along with binaries/scripts that take YAML input, and output parser events, JSON, native data or YAML.

When you run the editor, it starts a multi-pane vim session inside of the docker container and a grid of chosen output windows. With a couple of useful vim mappings you can play around and immediately see the different outputs as you type in YAML.

YAML Test Matrix

Last year I took care of some old patches for YAML.pm, also wrote some new. Then I realized that I wanted to write a parser/loader myself. It's now on CPAN called YAML::PP and is still experimental.

I had the wish to see all tests results from the Test Suite for every framework supported by the YAML editor, so I wrote code to run the tests, analyse the output and create a big HTML table. In this view you can easily spot edge cases.

All of these three projects helped me a lot to implement my parser, and I probably would have never started it without. It's fun working with the editor and implement a parser from test cases. It's also a good excuse to not read the spec ;-)

Perl Toolchain Summit Thanks

For the Perl Toolchain Summit, I would like to thank the organizers and helpers, Neil Bowers, Philippe Bruhat, Laurent Boivin and Wendy for providing us hotel rooms, a comfortable work environment and healthy (and less healthy) food.

This wouldn't be possible without the sponsors, thank you:

Booking.com, ActiveState, cPanel, FastMail, MaxMind, Perl Careers, MongoDB, SureVoIP, Campus Explorer, Bytemark, CAPSiDE, Charlie Gonzalez, Elastic, OpusVL, Perl Services, Procura, XS4ALL, Oetiker+Partner.