YAML and more at the Perl Toolchain Summit 2019
This year, I was happy to hear I would be invited again to the Perl Toolchain Summit, an annual event with about 30-35 people hacking four days on toolchain related stuff, improving user experience.
This time it was held in Marlow, a small city two hours from London, in an old abbey which was beautiful (but cold!).
My main goal was to get a big step closer to make YAML::PP more compatible to existing YAML modules.
You should now (v0.013) be able to use it for many cases as a drop-in replacement for YAML.pm, YAML::XS or YAML::Syck. Version 0.014 is on its way but there are CPAN infrastructure problems.
Generic Perl Objects and Types
In the weeks before the summit I had already implemented loading and dumping of generic perl objects and types (scalar references, regexes, code, ...). This is how you use it:
my $yp = YAML::PP->new( schema => [qw/ JSON Perl /] );
For convenience, you can also use YAML::PP::Perl:
my $yp = YAML::PP::Perl->new;
Dumping code references is on by default, as it is not dangerous. (Well, please let me know if it is...).
You can activate (dangerous) loading code like this:
my $yp = YAML::PP->new( schema => [qw/ JSON Perl +loadcode /] );
Now one issue was that YAML.pm, YAML::XS and YAML::Syck used the tag
!!perl... for that, while the YAML spec rather recommends using
!perl..., as two exclamation marks are a shorthand for the official
YAML namespace. Python for example uses
See Support both
So I decided to support both and let users choose.
With the next version you can decide which tag type you want to use. You can also use both for loading and one for dumping.
my $yp = YAML::PP->new( schema => [qw/ JSON Perl tag=!!perl+!perl /], );
!perl/typetags are supported. (default)
!!perl/typetags are supported.
!!perl/tagare supported when loading. When dumping,
!!perl/tagare supported when loading. When dumping,
Additionally, all supported types are documented clearly in the EXAMPLES section. I use this data for testing and generate the documentation from it to make sure everything is documented at all times.
Please note that in version 0.013 the tag style notation looked different. I wanted to upload version 0.014 but currently something in the CPAN infrastructure seems broken, so no new modules are showing up on MetaCPAN.
How to serialize Binary Data in YAML?
This is possible by encoding the binary data with Base64. A special tag
!!binary exists for that. It's not explicitly mentioned in the YAML 1.2 Spec.
It's listed in the Type Repository under Binary Data
Language-Independent Type for YAML™ Version
canonical: !!binary "\ R0lGODlhDAAMAIQAAP//9/X17unp5WZmZgAAAOfn515eXvPz7Y6OjuDg4J+fn5\ OTk6enp56enmlpaWNjY6Ojo4SEhP/++f/++f/++f/++f/++f/++f/++f/++f/+\ +f/++f/++f/++f/++f/++SH+Dk1hZGUgd2l0aCBHSU1QACwAAAAADAAMAAAFLC\ AgjoEwnuNAFOhpEMTRiggcz4BNJHrv/zCFcLiwMWYNG84BwwEeECcgggoBADs=" description: The binary value above is a tiny arrow encoded as a gif image.
You can use this by adding the
my $yp = YAML::PP->new( schema => [qw/ JSON Binary /] );
It also encodes binary data when dumping. It looks if there are any non-ASCII characters in a string, and if it is utf8 decoded or not. Please try it out and let me know if it is encoding data which shouldn't be encoded.
There is more stuff that I did on YAML::PP and plan to do soon. I will blog about that seperately.
The modules are still not completely compatible, for example regarding
how things like
TRUE, etc. are loaded.
YAML.pm, YAML::XS and YAML::Syck never really implemented the official types and also are different to each other, so it's hard to provide a compatible drop-in replacement. Not complicated, but a lot of typing and testing.
I did a security fix to YAML.pm. You
(hopefully) know that you shouldn't use the default YAML loader on untrusted
$YAML::LoadBlessed is true by default. The exploit is tricky
and might not work in every case. However, there's a much simpler exploit which
lets you set any package variable by loading a specially crafted YAML file:
--- !!perl/glob PACKAGE: YAML NAME: LoadCode SCALAR: 1 EOM
You can probably guess that after loading this YAML, loading code is
activated, which involves an
Please update YAML.pm to 1.28, where you have to enable glob loading via
$YAML::LoadCode option itself.
When I stumbled upon this last year, I made it already safer by only loading
$YAML::LoadBlessed is set, but that is still try by default,
and actually it isn't related to
There were some issues and Pull Requests for Inline.pm. I reviewed, quickly checked with Ingy and made a new release.
MetaCPAN command line client
I started a little command line client which uses MetaCPAN::Client.
It uses my App::Spec framework. It uses subcommands and completion and I hope it will make it easier to play around with the MetaCPAN API.
So far it is only on GitHub.
Thanks to the organizers and sponsors for this Summit!
Everything went smooth, which is usually a result of hard working organizers, Neil, BooK and Laurent.
Special thanks to Neil for baking us vegan blueberry muffins!
One thing I always look forward to is the excellent catering by Wendy. A lot of healthy vegetables (and not so healthy sweets). I like eating raw vegetables, but at home I'm often too lazy to prepare it. Wendy went shopping, cut and peeled stuff and also took care for vegan needs, for example. ❤️❤️❤️