Method Privacy in Perl

This is a slightly expanded version of a comment I posted a couple of days ago on NEILB's blog.

Neil was mostly talking about private functions, while I'll be talking mostly about private methods (i.e. object-oriented programming), but I think there's probably a good deal of overlap between the two concepts.

The traditional way of indicating a method is private is to name it with a leading underscore:

   sub _secret_sauce {
      my $self = shift;
      ...;
   }
   
   $self->_secret_sauce(@args);

The assumption is even baked into the Perl development and testing toolchain in a few places. For example Pod::Coverage won't complain about missing documentation for a sub if that sub happens to be named with a leading underscore.

I used to think the underscore convention was good enough. But partly because of this upcoming project and partly because of problems I've encountered working on various codebases, I've been forced to re-evaluate my thinking, and have come to the conclusion that the underscore convention is insufficient.

Recently I was writing a subclass of somebody else's class and noticed it suddenly started behaving strangely. After an embarrassing amount of time spent debugging I discovered that one of my underscore methods happened to have the same name as one of the superclass' underscore methods (actually two levels up inheritance), so was getting called by the superclass when I hadn't expected it to be.

Of course, if the underscore method in the superclass had been documented, the name collision could have been easily avoided - I could have just called my underscore method something else (which I did in the end). But nobody bloody documents the underscore methods! (Cheers, Pod::Coverage!)

As an aside: no, roles do not completely solve this problem.

There's a famous Larry Wall quote that Perl "would prefer that you stayed out of its living room because you weren't invited, not because it has a shotgun." Perhaps I'm stretching the analogy a little, but if there are no walls around anybody's living room, it's quite easily to accidentally cross into somebody else's living room. By the time you have noticed this, you might have already started furnishing it.

Having studied .NET a bit recently, I've come to be a big fan of the four levels of method privacy it offers:

  • Private methods may be called within their own class, and cannot be overridden by subclasses.
  • Family (a.k.a. Protected) methods may be called by their own class and any subclasses or superclasses, and may be overridden by subclasses.
  • Friend (a.k.a. Internal) methods may be called anywhere within the same assembly, and may be overridden by subclasses within the same assembly. (What's an assembly? Probably the nearest analogy in the Perl world would be a CPAN distribution; i.e. a collection of related modules all maintained by the same person/team. In .NET it has a more formal meaning, and variables, functions, etc can be scoped by assembly.)
  • Public methods may be called by anyone, of course, and may be overridden by subclasses.

.NET also supports sort of Friend Or Family and Friend And Family levels of privacy, though not all .NET programming languages expose those features of the underlying runtime. (C# for example supports Friend Or Family, but not Friend And Family.)

Back to Perl. How and why can we apply these ideas to Perl classes and roles?

  • Private methods can be emulated with coderefs stashed in lexical variables:
       my $_secret_sauce = sub {
          my $self = shift;
          ...;
       };
       
       $self->$_secret_sauce(@args);

    Assuming you define one package per file, or at least put some curly braces around each package definition, this method will not be accessible outside your package. In fact, you can even carefully scope it to be visible only to certain parts of your package, though for smallish classes that's overkill.

    If I write private methods this way, I can be confident I'm not going to acidentally overwrite a method in a superclass, and no subclass will accidentally overwrite mine either.

  • Family methods are those that you don't want to make part of the public API, but could be useful for subclasses to be able to call, or override. For these stick with the underscore convention, but try to make an effort for these methods to be documented, and preferably as stable as your public API.

    They don't need to clutter up the main documentation - you could write a separate "advice for subclassers" bit of documentation. Just a list of the names of these methods is probably sufficient - it will give subclassers a hint for what subs to search for in your source code.

  • Friend methods are more interesting. I'm going to propose a brand new convention for these. Say you're working on a project called "Aardwolf", then pick a related prefix name, such as "_aardwolf_" and use that prefix on all these methods:

       sub _aardwolf_secret_sauce {
          my $self = shift;
          ...;
       }
       
       $obj->_aardwolf_secret_sauce(@args);

    They don't need to be documented. Rather, they need to be urinated on to mark your territory. A note like this, either in your pod, or as a source code comment should clarify their status:

       # All methods with the "_aardwolf_" prefix are not part of the public
       # API, are not documented, and should not be considered stable. They
       # may change at any time with zero notice period. If you are not me,
       # do not override them, and do not call them. They are mine! Not yours!

    You can call these methods from outside your class - from entirely different parts of your project. You can do that safe in the knowledge that the entire thing is one big project, and if you need to change one of these methods, then you will also be able to fix up all the places which call that method. (Because they're all in the same source code repository, and maintained by the same person/people, and released according to the same schedule, etc, etc.)

    If somebody else is working on a different project - the RobotPet project - and wants to subclass your Aardwolf class as Robot::Aardwolf, then they're "allowed" to call and override your public API (of course), as well as your Family (single underscore convention) methods. But they're "not allowed" to call or override your Friend ("_aardwolf_") methods.

    Of course, they can define their own Friend ("_robotpet_") methods, and the naming convention guarantees that they will never interfere with each other!

    Another aside: "allowed", "not allowed" - this is where the living room analogy comes back into play.

  • Public methods I don't think I need to go into. You know this already.

    Document them of course. Otherwise how is everybody supposed to know how to use your objects?

Careful class design seems to make Friend and Family methods rarely necessary - it's often possible to promote or demote them to Public or Private methods.

So anyway, think about these ideas when you're next writing object-oriented code. Even if you don't adopt the conventioned I've described above, clarifying in your own mind who should be "allowed" to call and override your methods can lead you to a better class design, and better documentation.

11 Comments

I think this obviously states a problem that perhaps needs to get propagated into perl 6 design, methods should /have/ to designate the desire to override, or otherwise result in a conflict (like 2 roles being imported do)

I think private methods are mostly a bad idea, they violate, imho, the Open Closed principle, by making your class less open for extension. A subclass /should/ be able to override a method, though it seems like a good idea that this is explicit so that you don't accidentally violate Liskov Substitution (which is what you were doing).

A reason for privacy making code more confusing is, where you have say 2 private methods of the same name in the hierarchy. This is more apparent with field hiding than methods, where your class has private field foo, and the parent has a protected field foo, now go look at implementation, now subclass once more, and look at references of foo, and start thinking about which foo it's referencing, the answer is obvious, but sometimes in reading the code and skipping around the class it starts to seem less so.

when I say private methods, I mean java/c# style, perl private methods are actually more like protected, or subclass accessible, which doesn't violate Open Closed. in other words $self->_foo() is always ok, but you should never call $myref->_foo()

I've also been burned by overriding a "private" method in a parent class. Class::Method::Modifiers provides the fresh modifier, which will throw an exception if you override a method already defined by a parent class.

I don't think there is a problem to begin with. If this thing is so private, is it really a method? Just call it as a function and call it a day:

sub _secret_sauce {
    my $self = shift;
    ...;
}
   
_secret_sauce($self, @args);

...You're welcome! :)

Sure, it does not solve the non-private method classification and designation problem. It does, however, solve the private method problem. What one needs to observe is that if a method is so private that it cannot be called from any other class, it does not have to be a method. In other words, there is no need for private methods at all.

Maybe the desire for private methods is due to once-upon-a-time carbon-copy of Java concepts into Perl. The problem that Java has is that there are no functions, everything is a method (I think -- don't claim to be a Java expert). We, on the other hand, don't have this problem and can define and call functions as we see fit.

I've been wondering recently if it would be possible to use the MRO API to be able to invoke lexical subs as methods. Then you wouldn't have to use the $ sigil in your method calls.

Don't use inheritance. All problems solved.

I got really caught off guard by:

They don't need to be documented. Rather, they need to be urinated on to mark your territory.

The post was heavy then I hit that line and had to laugh. Thanks, good way to start off the day

Leave a comment

About Toby Inkster

user-pic I'm tobyink on CPAN, IRC and PerlMonks.