Buddy Burden

[What I really should have done here is something like Tom Wyant or Joel Berger.  Being a literary type, that appeals to me.  Of course, with Dupin and Holmes taken, who would I pay homage to? Nero Wolfe, perhaps? Batman? Michael Westen? Harry Dresden?

Anyways, maybe next time, if I have more time to prepare.  For now, you’ll have to settle for plain ol’ me.]

At $work, I’m the guy who maintains the tool we use for branching, merging, pushing ... release management, I suppose you’d say.  I don’t actually do the pushes most of the time, but I’m usually the guy who takes the tickets when there’s something wrong with the process.

At the end of the push, we build a new AMI.  That’s Amazon-speak for a master virt which we can clone to bring up new production machines quickly.  By making a new one after every push, any new instances we bring up will automatically have the latest code on them.  Recently, my coworker doing the push reported that the AMI build failed.  Well, he clarified: it says it failed, although it actually looked like it succeeded.  Probably a transient error, I assured him (with fingers crossed).  But, no such luck: the next week, the same thing happened.  This time my coworker figured out what underlying command the release management script was running and ran it by hand.  Yep, it definitely succeeds, he reported.  It just says it failed.

And so it went for the next three pushes.  You know how it is ... it’s irksome, but there’s nothing actually broken, so why waste time fiddling with it?  Finally someone decided to make a ticket for it.  I spoke up in tech chat: you know guys, probably someone changed something with the underlying command so it doesn’t exit cleanly.  The poor release management script is stupid; all it does is run the command, then check the exit value.  Non-zero, report failure.  I bet anyone could fix this with a bit of time and some Git archaeology.

Two more pushes with people grumbling about it and I finally just grabbed the ticket myself.  I found the exact date of the first push where it happened (and that’s why you always log your tech chat, people), then started poking around the Git commit logs for any changes just before that date.

As it turns out, another coworker had decided to refactor the command entirely, moving the main logic out of the bin/ script and into its own class.  Well, nothing wrong with that, I thought ... sounds like a pretty good idea to me.  The guts of the script was reduced to dealing with configuration, environment, and options, and then this code:

my $command = CE::Command::ec2_launch_instance->new(
    # bunches o' parameters here
);
$command->execute();
exit $command->success ? 0 : 1;

Well, nothing much to see there, I thought.  Better look where all the action is and see what’s really going on:

sub execute {
    my $self = shift;
    # do some stuff
    return unless $self->wait;
    # more stuff, including creating $manager
    if (not $manager->all_success) {
        $self->success(0);
        return;
    }
    # a wee bit more stuff
    $self->success(1);
}

Hunh, I thought.  I still don’t see it.

Do you?  I’m sure some of you out there spied it instantly and are even now nodding your heads in commiseration.  If you didn’t spot it though, go back and look.  I promise you, I gave you enough of the code to figure it out ... nothing I left out is relevant to the problem.  Go ahead and ponder it for a bit.  I’ll wait.

Sure you want to peek at the answer?

Really sure?

Okay, I’ll tell you what it was.

I was all set for a marathon debugging session ... I figured I’d have to run the whole stupid script a bunch of times, adding more and more debugging.  Or maybe (<shudder>) actually fire up the debugger.^*  Which wouldn’t be so bad, except building an AMI takes a fair amount of time, so it’s not the sort of thing you really want to be doing over and over again until you winkle out an obscure bug.  Especially for a bug where nothing’s actually broken.

But then I had a nagging hunch.  I wish I could tell you what triggered it, but I honestly don’t know.  Probably something I read somewhere a long time ago ... and this is why you must read about Perl (or whatever your language-of-choice is).  Read the manuals, read the books, read the blogs.  Go download Modern Perl.  Subscribe to Perl Weekly.  The more you know, the better chance that, when you hit that weird bug, something’s going to tickle the back of your brain, something’s going to stick in the sewer grate at the bottom of your mind, something’s going to blink a little light way off at the periphery of your vision, and the little light will say ...

Wait.  Go back a bit.  What was the middle thing?

Although my forebrain told me to concentrate on the execute method—that’s where all the action is, after all—something made me glance back at the script itself.  Remember this part?

my $command = CE::Command::ec2_launch_instance->new(
    # bunches o' parameters here
);
$command->execute();
exit $command->success ? 0 : 1;

Even more specifically ... this part?

exit $command->success ? 0 : 1;

Something wrong here ... maybe exit can’t take an expression?

[absalom:~] perldoc -f exit
       exit EXPR
       exit    Evaluates EXPR and exits immediately with that value.    Example:
       :
       :

Well, so much for that theory.  Still ... something ...

Could this be one of Perl’s weird precedence things?  I mean, I know ternaries are pretty low in the precedence list ... let’s look:

[absalom:~] perldoc op
PERLOP(1)                                  User Contributed Perl Documentation                                  PERLOP(1)
 
 
 
NAME
       perlop - Perl operators and precedence
 
DESCRIPTION
   Operator Precedence and Associativity
       Operator precedence and associativity work in Perl more or less like they do in mathematics.
       :
       :
 
           left        terms and list operators (leftward)
           left        ->
           nonassoc    ++ --
           right       **
           right       ! ~ \ and unary + and -
           left        =~ !~
           left        * / % x
           left        + - .
           left        << >>
           nonassoc    named unary operators
           nonassoc    < > <= >= lt gt le ge
           nonassoc    == != <=> eq ne cmp ~~
           left        &
           left        | ^
           left        &&
           left        || //
           nonassoc    ..  ...
           right       ?:
           right       = += -= *= etc.
           left        , =>
           nonassoc    list operators (rightward)
           right       not
           left        and
           left        or xor
 
       :
       :

What’s that thing roughly in the middle there? “named unary operators”?  What are the chances, do you suppose, that exit is a “named unary operator”?

Pretty good, as it happens.

[absalom:~] perl -le 'exit 2' ; echo $?
2
[absalom:~] perl -le 'exit 1 ? 2 : 3' ; echo $?
1

Crap.

You mean this

exit($command->success ? 0 : 1);

would fix it?

Yep.  That fixes it.

Crap.

Well, lesson learned and all that, and maybe this will help someone else out there, but I also wonder if maybe the man page could be clarified a bit.  I can’t help but think that this:

[absalom:~] perldoc -f exit
       exit EXPR

is a bit misleading.  Sure, exit EXPR ... but only if EXPR consists only of terms and operators higher in the precedence list than “named unary operators.” That seems like a bit of a gotcha.

Anyways, mystery solved, and here’s my commit message:

Author: Buddy Burden <barefootcoder@gmail.com>
Date:   Wed Jun 4 15:06:49 2014
 
    oh look: named unary operators have a higher precedence than the ternary <sigh>

[P.S.: I hope to see many of you at YAPC in a week or so.  If you see a short dumpy guy wandering the halls with a scruffy beard and no shoes, walk over and say “hi!”]