Ovid

Today I wasted a few hours tracking down this delightful bug:

Undefined subroutine &main::main:: called at 
...lib/site_perl/5.12.4/YAML/Mo.pm line 5.

So what does YAML::Mo line 5 look like?

That's right. That's line 5. I've wrapped it to make it easier to read. When you load the latest YAML, you load YAML::Mo and that contains the above monstrosity. And it has a serious bug. Do you see it?

This bug was first reported a couple of months ago but no one could reproduce it. There was speculation that it was related to *{""} stringification, but while it's related, it's sort of a red herring.

I first encountered this bug when I was was using Dancer and was using the excellent Data::DPath (thank's Steffen!) to extract data from a JSON data structure. At first, Dancer was telling me that YAML was not installed, but a pull request later, it was giving me the error message I opened this post with. And what a heck of a problem it was to debug.

After a lot of work stripping down Data::DPath, I managed to get this test case:

use Safe;
BEGIN { Safe->new } # used in Data::DPath
use YAML;

As it turns out, there's a tiny bug in Safe's calculation of shared code and I've filed a bug report for it. Essentially, it does this:

\&main::;

That creates a reference to a sub whose name is the empty string! Actually, no it doesn't. It appears to autovivify the typeglob slot in the same way a forward declaration will, but it provides no sub body.

# a forward declaration of a sub, but no sub body
sub foo;

Now what does strict do? Here's some fun:

[Ovid:~] $ perl -le 'my $x = foo; print $x'
foo
[Ovid:~] $ perl -le 'sub foo { "bar" } my $x = foo; print $x'
bar

If you don't enable strict, Perl will see if a bareword is a subroutine and if it is, it will call it. Otherwise, it will assume that the bareword is a string.

Actually, it's use strict 'subs' that does this:

[Ovid:~] $ perl -le 'use strict 'subs'; my $x = foo; print $x'
Bareword "foo" not allowed while "strict subs" in use at -e line 1.
Execution of -e aborted due to compilation errors.

At the beginning of the YAML::Mo code, we have this:

package YAML::Mo;
my $M = __PACKAGE__ . ::;

Because strict is not enabled, $M will be assigned the value YAML::Mo::. Clever, huh? No, not really. When Safe is loaded, the reference \&main:: in Safe will create a subroutine effectively named "" (the empty string). When you reference a package variable or subroutine, you're allowed to leave off the main:: if it's in package main, so if you have this:

package main;
sub foo { 'bar' }

The rest of your code can call foo() with either main::foo() or ::foo(). In this case, here's what happens:

• Perl sees a bare :: in __PACKAGE__ . ::;
• It first tries to see if :: is a sub, but since main is optional, it sees :: as a sub in main::.
• Because of the Safe bug, there is now an empty string sub in *main::{CODE}, but it has no sub body.
• So Perl tries to call *main::, resulting in the strange Undefined subroutine &main::main:: error.

Here's the smallest test case to replicate the bug:

[Ovid~] $ perl -e 'BEGIN { \&:: } use Mo'
Undefined subroutine &main::main:: called at 
... lib/site_perl/5.12.4/Mo.pm line 3.

So now, any code that used Mo to generate "inline Mo" code needs to be rereleased to work around this bug because updating Mo is not enough. You must update the code that inlines Mo.

The fundamental problem here isn't actually the lack of using strict (though this problem would not have happened if it was used). It's trying to be too clever. I have often maintained two things:

• Clever programmers often aren't.
• Expert programmers try to make their code as clean and simple as possible.

I have no problem with "clever" code, but only as a proof of concept, for an Acme:: module, or when you have no choice. Willy-nilly applying this to YAML was not a good decision (Sorry Ingy. I luv ya, but honestly ...).

So everyone using YAML now has a potential time bomb in their code. If they load Safe first, or any other code that accidentally creates subs in namespaces where "Mo" thinks there should not be subs, they could fall afoul of this.

And for your amusement, here's a bug report whereby someone gets a commit bit for golfing Mo even further.

And this is the bug report for YAML.

Please don't be clever.