Why I wrote Keyword::DEVELOPMENT

I've had a few discussions with people about why I wrote Keyword::DEVELOPMENT, a useful, but simple module. In short, it lets you do this:

use Keyword::DEVELOPMENT;

# later ...

sub consume_item ( $self, $item_slug ) {
    DEVELOPMENT {
      # expensive and time-consuming debugging code here
    }
    my $inventory = $self->inventory;
    return $self->new_exchange(
        slug => 'consume',
        Steps(
            Inventory(  $inventory => contains => $item           ),
            Consumable( $self      => consume  => 'item_instance' ),
            Inventory(  $inventory => remove   => 'item_instance' ),
        ),
    )->attempt;
}

The expensive debug block? With Keyword::DEVELOPMENT, is simply fails to exist in production unless the PERL_KEYWORD_DEVELOPMENT environment variable has a true value. Thus, there is no performance overhead to instrumenting your code with complicated debugging logic (as I'm currently doing with Tau Station).

It's been pointed out to me by several people that I can accomplish the above in regular Perl with constant folding. I'll explain what that means and why I prefer the keyword route.

I prefer this module to constant folding because Keyword::DEVELOPMENT is less fragile (assuming you're comfortable with pluggable keywords: they have been available for about 7 years now) and it creates a standard in our codebase.

With constant folding, you can do the following:

#!/usr/bin/env perl

use 5.024;
use constant DEBUG => $ENV{DEBUG_MY_LOUSY_CODE};

# in exchange...
if (DEBUG) {
    say "Yes, debug!";
}
else {
    say "No, debug!";
}

In the above, if the DEBUG_MY_LOUSY_CODE environment variable doesn't exist, or has a false value, the Perl compiler will see that DEBUG is false, at compile time, and the if block will simply be removed. There won't even be a check for it. It's a powerful tool. Here's the output from perl -MO=Deparse const.pl:

sub BEGIN {
    require 5.024;
}
use constant ('DEBUG', 0);
use strict;
no feature ':all';
use feature ':5.24';
do {
    say 'No, debug!'
};
const.pl syntax OK

Note that the "Yes, debug!" code is not there. That's also what Keyword::DEVELOPMENT does. Except ...

Some dev comes along as says "why are we using the old constant pragma?" So they "fix" the code:

#!/usr/bin/env perl

use 5.024;
use Const::Fast;
const my $DEBUG => $ENV{DEBUG_MY_LOUSY_CODE};

# in exchange...
if ($DEBUG) {
    say "Yes, debug!";
}
else {
    say "No, debug!";
}

And in the deparse:

sub BEGIN {
    require 5.024;
}
use Const::Fast;
use strict;
no feature ':all';
use feature ':5.24';
&const(\my $DEBUG, 0);
if ($DEBUG) {
    say 'Yes, debug!';
}
else {
    say 'No, debug!';
}
const.pl syntax OK

So now we still have our if block there, causing a couple of extra ops which may be problematic in hot code.

And then there's this bit I don't understand at all. In the use constant code, if DEBUG is false, we get a do block instead of the if/else checks. Running perl -MO=Concise,-exec const.pl reveals this:

1  <0> enter
2  <;> nextstate(main 188 const.pl:7) v:*,&,{,$,268437504
3  <0> enter v
4  <;> nextstate(main 192 const.pl:11) v:*,&,$,268437504
5  <0> pushmark s
6  <$> const(PV "No, debug!") s
7  <@> say vK
8  <@> leave vKP
9  <@> leave[1 ref] vKP/REFC

Those are the ops we have with this code and represents what Perl sees internally. Note that this optimized code does not have the "Yes, debug!" code.

Here's the same code, but with DEBUG set to true, thus causing the "Yes, debug!" expression to be printed:

1  <0> enter
2  <;> nextstate(main 188 const.pl:7) v:*,&,{,$,268437504
3  <0> pushmark s
4  <$> const(PV "Yes, debug!") s
5  <@> say vK
6  <@> leave[1 ref] vKP/REFC

We don't get to leverage constant folding, but we actually have fewer ops! If you are running very hot code, shaving ops is important. If you take the else block off, you will get fewer opcodes with the constant folding. I don't actually know why this is, but there you go. I hope some internals guru will come along and tell me what I'm missing here :)

I see constant folding being used all the time for these types of optimizations, with constant names like DEBUG, TESTING, DEBUGGING, and so on. With Keyword::DEVELOPMENT:

  • You get a standard way of handing this
  • Developers thus learn to understand what is happening
  • You have less fragility due to not worrying about how constants are handled

It does require Perl 5 version 12 or above because it uses the pluggable keyword syntax, but it's dead simple:

package Keyword::Development;
use 5.012;   # pluggable keywords
use warnings;
use Keyword::Simple;

sub import {
    Keyword::Simple::define 'DEVELOPMENT', sub {
        my ($ref) = @_;
        if ( $ENV{PERL_KEYWORD_DEVELOPMENT} ) {
            substr( $$ref, 0, 0 ) = 'if (1)';
        }
        else {
            substr( $$ref, 0, 0 ) = 'if (0)';
        }
    };
}

sub unimport {
    Keyword::Simple::undefine 'DEVELOPMENT';
}

Having sane standards in your development shop makes everyone's life easier. Every time you can introduce a single way of accomplishing a task, it's less cognitive overhead for developers and means they spend less time wondering "what does that weird bit do?"

Naturally, you can fork the code from github, download it from the CPAN, or contact our company for training or building awesome software for you. Or just contact me at ovid at allaroundtheworld dot fr.

Note: we write awesome software in a variety of languages, not just Perl.

8 Comments

Some dev comes along as says "why are we using the old constant pragma?" So they "fix" the code:

Might I suggest not hiring people who will dis-improve the code by rote rule-following? 😊

But also, if that’s your problem:

use strict; use warnings;
package DEBUG;
sub import {
    eval sprintf 'sub %s::DEBUG () { %d } 1', ''.caller, !!$_[1]
        or die
}
1;

You can put ample warnings in the POD not to touch that file without talking about it with the dev lead.

This DEBUG.pm plays the exact same role as Keyword::DEVELOPMENT, except for not needing the keywords API or Keyword::Simple.

If you take the else block off, you will get fewer opcodes with the constant folding.

But how is that relevant to why Keyword::DEVELOPMENT is better? It only offers the equivalent of if (DEBUG) { ... } without an else block – a scenario in which both approaches yield the exact same opcodes.

It’s only the else block that gets the implicit-do{} treatment. You are complaining about the behaviour of the pure-Perl version for the equivalent of a feature your module doesn’t have in the first place.

I concede it’s not entirely irrelevant insofar as missing features cannot be misused, but it’s still an apples/oranges complaint.

I don't actually know why this is, but there you go. I hope some internals guru will come along and tell me what I'm missing here :)

Write the code with a variable to prevent constant-folding, and leave off the -exec switch when dumping the optree.

You’ll see that the whole if (...) {...} else {...} construct compiles to a ternary with two do{} blocks. But for some reason the one for the else branch gets to keep its nextstate op while the one for the if branch sees its nextstate op excised.

I don’t know why that difference exists. But my understanding is that the nextstate is the reason that that block cannot lose the enter/leave frame when the other branch is constant-folded away.

(I may have some of this backwards. One side is just a scope op while the other has an enter/leave frame, both of which amount to do{}, but maybe differently heavy versions. That much is the limit of my guts knowledge.)

I must say Const::Fast was poorly named.

I also think DEVELOPMENT is too long to type. I would have used DEBUG. But whatever.

Perl does not make a distinction between if/else and ?: at the op level. $cond ? do { this() } : do { that() } and if($cond){ this() } else { that() } give nearly identical op trees, which differ only incidentally, because they go through different code paths in Perl’s parser to get there. So that’s why you see do blocks in the deparse output after the effective ?: gets folded to just one of its branches.

Now the real reason is a lot more complicated and has some history to it. I don’t know the full answer offhand, but I do remember something like this: Some bug related to blocks or scopes was fixed between 5.10 and 5.12, but it caused a new bug, in that constant-folded if/elses no longer returned values (which matters at the end of a sub). So I fixed that bug by submitting a patch (I did not yet have a commit bit) to make the op tree of the folded block identical to that produced by do{}. My reasoning was simple. The folded blocks were not working properly; do blocks were working properly; so making the former identical to the latter just made things work. It was not necessary to dig deeper to find out why it was not working. (Back then I was just starting to poke around in the perl source. There was a lot I did not understand.) So that’s why you get do{} blocks after folding if-else or if(1). When folding a plain if(0), the block disappears entirely as always.

The difference between a scope op and an enter/leave pair is that the former is an optimised, ‘cheating’ version of the latter. A scope op does not actually do anything. It’s what perl uses when it realises that no scope is actually needed.

The reason why else doesn’t get this ‘fake scope’ treatment is that line numbers in errors and warnings will be off in the else block, and will point to the line of the if instead, which is sufficiently unhelpful to warrant foregoing the scope optimisation for else.

use Sub::Disable qw/DEV/; sub DEV (&) { (shift)->() } DEV { warn 123 };

No need for custom parser, but you'd need to put ; at the end of the block)

PS: Deparse will show you full code, but Concise runs optimizer stage, so you'll see the absence of the call site.

A nice change might be to support a list of env variable names to check, so that:

use Keyword::DEVELOPMENT qw(KEY1 KEY2);

would have the DEVELOPMENT keyword compile it's code if any of the listed env variables have a true value. That would permit you to turn on the dev code for subsets of the program. (Using a list instead of a single value allows turning on a groups of subsets. I'd always augment the user-provided list with PERLKEYWORDDEVELOPMENT to give a consistent way of turning on all of the dev code.

A nice change might be to support a list of env variable names to check, so that:

use Keyword::DEVELOPMENT qw(KEY1 KEY2);

would have the DEVELOPMENT keyword compile its code if any of the listed env variables have a true value. That would permit you to turn on the dev code for subsets of the program. (Using a list instead of a single value allows turning on a groups of subsets. The user-provided list could be augmented internally with PERLKEYWORDDEVELOPMENT to give a consistent way of turning on all of the dev code.

Leave a comment

About Ovid

user-pic Freelance Perl/Testing/Agile consultant and trainer. See http://www.allaroundtheworld.fr/ for our services. If you have a problem with Perl, we will solve it for you. And don't forget to buy my book! http://www.amazon.com/Beginning-Perl-Curtis-Poe/dp/1118013840/