p5-mop, a gentle introduction

I guess that you've heard about p5-mop by now.

If not, in a nutshell, p5-mop is an attempt to implement a subset of Moose into the core of Perl. Moose provides a Meta Object Protocol (MOP) to Perl. So does p5-mop, however p5-mop is implemented in a way that it can be properly included in the Perl core.

Keep in mind that p5-mop goal is to implement a subset of Moose. As Stevan Little says:

We are not putting "Moose into the core" because Moose is too opinionated, instead we want to put a minimal and less opinionated MOP in the core that is capable of hosting something like Moose

As far as I understood, after a first attempt that failed, Stevan Little restarted the p5-mop implementation: the so-called p5-mop-redux github project, using Devel::Declare, ( then Parse::Keyword ), so that he can experiment and release often, while keeping the implementation core-friendly. Once he's happy with the features and all, he'll make sure it finds its way to the core. A small team (Stevan Little, Jesse Luehrs, and other contributors) is actively developping p5-mop, and Stevan is regularly blogging about it.

If you want more details about the failing first attempt, there is a bunch of backlog and mailing lists archive to read. However, here is how Stevan would summarize it:

We started the first prototype, not remembering the old adage of "write the first one to throw away" and I got sentimentally attached to my choice of design approach. This new approach (p5-mop-redux) was purposfully built with a firm commitment to keeping it as simple as possible, therefore making it simpler to hack on. Also, instead of making the MOP I always wanted, I approached as building the mop people actually needed (one that worked well with existing perl classes, etc)

Few months ago, when p5-mop-redux was announced, I tried to give it a go. And you should too ! Because it's easy.

Why is it important to try it out ?

It's important to have at least a vague idea of where p5-mop stands at, because this project is shaping a big part of Perl's future. IMHO, there will be a before and an after having a MOP in core. And it is being designed and tested right now. So as Perl users, it's our chance to have a look at it, test it, and give our feedback.

Do we like the syntax ? Is it powerful enough? What did we prefer more/less in Moose ? etc. In few months, things will be decided and it'll only be a matter of time and implementation details. Now is the most exciting time to participate in the project. You don't need to hack on it, just try it out, and provide feedback.

Install it

p5-mop is very easy to install:

  1. you need at least perl 5.16. If you need to upgrade, consider perlbrew or plenv
  2. if you don't have cpanm, get it with curl -L http://cpanmin.us | perl - App::cpanminus
  3. first, we need to install twigils, with cpanm --dev twigils
  4. then p5-mop itself. If you're using github, just fork the p5-mop-redux project. Otherwise you can get a zip here.
  5. using cpanm, execute cpanm . from within the p5-mop-redux directory.

A first example

Here is the classical point example from the p5-mop test suite

use mop;

class Point {
    has $!x is ro = 0;
    has $!y is ro = 0;

    method set_x ($x) {
        $!x = $x;
    }

    method set_y ($y) {
        $!y = $y;
    }

    method clear {
        ($!x, $!y) = (0, 0);
    }

    method pack {
        +{ x => $self->x, y => $self->y }
    }
}

# ... subclass it ...

class Point3D extends Point {
    has $!z is ro = 0;

    method set_z ($z) {
        $!z = $z;
    }

    method pack {
        my $data = $self->next::method;
        $data->{z} = $!z;
        $data;
    }
}

This examples shows how straightforward it is to declare a class and a subclass. The syntax is very friendly and similar to what you may find in other languages.

class declares a class, with proper scoping. method is used to define methods, so no sub there. The distinction is important, because in methods, additional variables will be automatically available:

  • $self will be available directly, no need to shift @_.
  • attributes variable will be available automatically, so you can access attributes from within the class without having to use their $self->accessors.

Functions defined with the regular sub keyword won't have all these features, and that's for good: it makes the difference between function and method more explicit.

hasdeclares an attribute. Attribute names are twigils. Borrowed from Perl6, and implemented by Florian Ragwitz in its twigils project on github, twigils are useful to differenciate standard variables from attributes variables:

class Foo {
    has $!stuff;
    method do_stuff ($stuff) {
        $!stuff = $stuff;
    }
}

As you can see, it's important to be able to differenciate stuff (the variable) and stuff (the attribute).

The added benefit of attributes variables is that one doesn't need to contantly use $self. A good proportion of the code in a class is about attributes. Being able to use them directly is great.

Other notes worth mentiong:

  • Classes can have a BUILD method, as with Moose.
  • A class can inherit from an other one by extend-ing it.
  • In a inheriting class, calling the parent method is not done using SUPER, but $self->next::method.
  • A class Foo declared in the package Bar will be defined as Bar::Foo.

Attributes traits

When declaring an attribute name, you can add is, which is followed by a list of traits:

has $!bar is ro, lazy = $_->foo + 2;
  • ro / rw means it's read-only / read-write
  • lazy means the attribute constructor we'll be called only when the attribute is being used
  • weak_ref enables an attribute to be a weak reference

Default value / builder

has $!foo = 'default value';

which is actually

has $!foo = sub { 'default value' };

So, there is no default value, only builders. That means that has $!foo = {}; will work as expected ( creating a new hashref each time ).

You can reference the current instance in the attribute builder by using $_:

has $!foo = $_->_init_foo;

There has been some comments about using = instead of // or || or default, but this syntax is used in a lot of other programing language, and considered somehow the default (ha-ha) syntax. I think it's worth sticking with = for an easier learning curve for newcomers.

Class and method traits

UPDATE: Similarly to attributes, classes and methods can have traits. I won't go in details to keep this post short, but you can make a class abstract, change the default behaviour of all its attributes, make it work better with Moose, etc. Currently there is only one method trait to allow for operator overloading, but additional ones may appear shortly.

Methods parameters

When calling a method, the parameters are as usual available in @_. However you can also declare these parameters in the method signature:

method foo ($arg1, $arg2=10) {
    say $arg1;
}

Using = you can specify a default value. In the method body, these parameters will be available directly.

Types

Types are not yet core to the p5-mop, and the team is questioning this idea. The concensus is currently that types should not be part of the mop, to keep it simple and flexible. You ought to be able to choose what type system you want to use. I'm particularly happy about this decision. Perl is so versatile and flexible that it can be used (and bent to be used) in numerous environment and configuration. Sometimes you need robustness and high level powerful features, and it's great to use a powerful typing system like Moose's one. Sometimes (most of the time? ) Type::Tiny (before that I used Params::Validate) is good enough and gives you faster processing. Sometimes you don't want any type checking.

Clearer / predicate

Because the attribute builder is already implemented using =, what about clearer and predicate?

# clearer
method clear_foo { undef $!foo }

# predicate
method has_foo { defined $!foo }

That was pretty easy, right? Predicates and clearers have been introduced in Moose because writing them ourselves would require to access the underlying HashRef behind an instance (e.g. sub predicate { exists $self->{$attr_name}}) and that's very bad. To work around that, Moose has to generate that kind of code and provide a way to enable it or not. Hence the predicateand clearer options. So you see that they exists mostly because of the implementation.

In p5-mop, thanks to the twigils, there is no issue in writing predicates and cleare ourselves.

But I hear you say "Wait, these are no clearer nor predicate ! They are not testing the existence of the attributes, but their define-ness!" You're right, but read on!

Undef versus not set

In Moose there is a difference between an attribute being unset, and an attribute being undef. In p5-mop, there is no such distinction. Technically, it would be very difficult to implemente that distinction, because an attribute variable is declared even if the attribute has not been set yet.

In Moose, because objects are stored in blessed hashes, an attribute can either be:

  • non-existent in the underlying hash
  • present in the hash but with an undef value
  • present and defined but false
  • present, defined and true

That's probably too many cases... Getting rid of one of them looks sane to me.

After all, we got this "not set" state only because objects are stored in HashRef, so it looks like it's an implementation detail that made its way into becoming a concept on its own, which is rarely a good thing.

Plus, in standard Perl programming, if an optional argument is not passed to a function, it's not "non-existent", it's undef:

foo();
sub foo {
    my ($arg) = @_; # $arg is undef
}

So it makes sense to have a similar behavior in p5-mop - that is, an attribute that is not set is undef.

Roles

Roles definition syntax is quite similar to defining a class.

role Bar {
    has $!additional_attr = 42;
    method more_feature { say $!additional_attr }
}

They are consumed right in the class declaration line:

class Foo with Bar, Baz {
    # ...
}

Meta

Going meta is not difficult either but I won't describe it here, as I just want to showcase default OO programming syntax. On that note, it looks like Stevan will make classes immutable by default, unless specified. I think that this is a good idea (how many time have you written make_immutable ?).

My (hopefully constructive) remarks

Method Modifiers

Method modifiers are not yet implemented, but they won't be difficult to implement. Actually, here is an example of how to implement method modifiers using p5-mop very own meta. It implements around:

sub modifier {
    if ($_[0]->isa('mop::method')) {
        my $method = shift;
        my $type   = shift;
        my $meta   = $method->associated_meta;
        if ($meta->isa('mop::role')) {
            if ( $type eq 'around' ) {
                $meta->bind('after:COMPOSE' => sub {
                    my ($self, $other) = @_;
                    if ($other->has_method( $method->name )) {
                        my $old_method = $other->remove_method( $method->name );
                        $other->add_method(
                            $other->method_class->new(
                                name => $method->name,
                                body => sub {
                                    local ${^NEXT} = $old_method->body;
                                    my $self = shift;
                                    $method->execute( $self, [ @_ ] );
                                }
                            )
                        );
                    }
                });
            } elsif ( $type eq 'before' ) {
                die "before not yet supported";
            } elsif ( $type eq 'after' ) {
                die "after not yet supported";
            } else {
                die "I have no idea what to do with $type";
            }
        } elsif ($meta->isa('mop::class')) {
            die "modifiers on classes not yet supported";
        }
    }
}

It is supposed to be used like this:

method my_method is modifier('around') ($arg) {
    $arg % 2 and return $self->${^NEXT}(@_);
    die "foo";
}

I would like to see method modifiers in p5-mop. As per Stevan Little and Jesse Luehrs, it may be that these won't be part of the mop, but in a plugin or extension. I'm not to sure about that, for me method modifier is really linked to OO programmning. I prefer using around than fiddling with $self->next::method or ${^NEXT}.

Here are some syntax proposals I've gathered on IRC and blog comments regarding what could be method modifiers in p5-mop:

around foo { }
method foo is around { ... }
method foo is modifier(around) { ... }

${^NEXT} and ${^SELF}

These special variables are pointing to the current instance (useful when you're not in a method - otherwise $self is available), and the next method in the calling chain. It's OK to have such variables, but their horrible name makes it difficult to remember and use.

Can't we have yet an other type of twigils for these variables ? so that we can write $^NEXT and $^SELF.

Twigils for public / private attributes

Just an idea, but maybe we could have $!public_attribute and $.private_attribute. Or is it the other way around ?

why is ? we already have has !

This one thing is bothering me a lot: why do we have to use the word is when declaring an attribute? The attribute declaration starts with has. So with is, that makes it two verbs for one line of code. For me it's too much. in Moo* modules, the is was just one property. We had default, lazy, etc. Now, is is just a seperator between the name and the 'traits'. In my opinion, it's redundant.

Also, among the new keywords added by p5-mop, we have only nouns (class, role, method). Only one verb, has.

The counter argument on this is that this syntax is inspired by Perl6:

class Point is rw {
    has ($.x, $.y);
    method gist { "Point a x=$.x y=$.y" }
}

So, "blame Larry" ? :)

Exporter

p5-mop doesn't use @ISA for inheritance, so use base 'Exporter' won't work. You have to do use Exporter 'import'. That is somewhat disturbing because most Perl developers (I think) implement functions and variables exporting by inheriting from Exporter (that's also what the documentation of Exporter recommends).

You could argue that one should code clean classes (that don't export anything, and clean modules (that export stuff but don't do OO). Mixing OO in a class with methods and exportable subs looks a bit un-orthodox. But that's what we do all day long and it is almost part of the Perl culture now. Think about all the modules that provides 2 APIs, a functional one and an OO one. All in the same namespace. So, somehow, being able to easily export subs is needed.

However, as per Jesse Luehrs and Stevan Little, they don't think a MOP implementation should be in charge of implementing an Exporter module, and I can totally agree with this. So it looks like the solution will be a method trait, like exportable:

sub foo is exportable { ... }

But that is not yet implemented.

Inside Out objects versus blessed structure objects

p5-mop is not using the standard scheme where an object is simply a blessed structure (usually a HashRef). Instead, it's using InsideOut objects, where all you get as an object is some kind of identification number (usually a simple reference), which is used internally to retrieve the object properties, only accessible from within the class.

This way of doing may seem odd at first: if I recall correctly, there a time where InsideOut objects were trendy, especially using Class::Std. But that didn't last long, when Moose and its follow ups came back to using regular blessed structured objects.

The important thing to keep in mind is that it doesn't matter too much. Using inside out objects is not a big deal because p5-mop provides so much power to interact and introspect with the OO concepts that it's not a problem at all that the attributes are not in a blessed HashRef.

However, a lot of third-party modules assume that your objects are blessed HashRef. So when switching to p5-mop, a whole little ecosystem will need to be rewritten.

UPDATE: ilmari pointed out in the comments that there is a class trait called repr that makes it possible to change the way an instance is implemented. You can specify if an object should be a reference on a scalar, array, hash, glob, or even a reference on a provided CodeRef. This makes p5-mop objects much more compatible with the OO ecosystem.

Now, where to ?

Now, it's your turn to try it out, make up your mind, try to port an module or write on from scratch using p5-mop, and give your feedback. To do that, go to the IRC channel #p5-mop on the irc.perl.org server, say hi, and explain what you tried, what went well and what didn't, and how you feel about the syntax and concepts.

Also, spread the word by writing about your experience with p5-mop, for instance on blogs.perl.org.

Lastly, don't hesitate to participate in the comments below :) Especially if you don't agree with my remarks above.

Reference / See also

Contributors

This article has been written by Damien Krotkine, but these people helped proof-reading it:

  • Stevan Little
  • Jesse Luehrs
  • Toby Inkster
  • Lukas Atkinson

18 Comments

I've installed and tried it, too - works nicely.

"Can I have it?" "Are we there yet?" "Will it be in 5.18.2?" "Can I have it with 5.20?" "Are we there yet?" ... :)

Please please really add it to "core", I'm begging you. :)

There is a 'repr' trait that lets you specify which type of reference to use for the object ID, so you can extend non-mop classes. You'd have to use accessors for inherited attributes, though.

I don't think I like it. Moose in core I think is a good idea, however I don't think MooseX::Declare style stuff needs be in core.

If this stuff gets in core, then you've effectively destroyed backcompat.

That said, a metaobject model and standardized accessors in core with Moose/Moo for backcompat would be a good thing.

TL;DR I don't like the new fandangled fancy class/method/is stuff, I'll not be using it, and I won't be using anyone elses code that uses it, same as I won't use anything that uses MX::D at the moment.

5.18.2 will be mostly bug fixes, so it's not going to be in there.

5.20 is only 8 months away, so it seems very unlikely.

The current implementation is built on some pretty fragile techniques - these have been necessary to get it to work purely through modules, with no hooks added to Perl itself.

In particular, for implementing the mop's keywords (class, method, etc) it would be nice if the keyword API (introduced in Perl 5.14) played well with lexical subs (an experiment from Perl 5.18) so that the keywords could be used without polluting into the caller's namespace.

And the twigil implementation relies on the parsing of the Perl built-in special global $!, looking for a bareword following it. It would be nice if there were proper support for parsing twigils in Perl core, even if that was just to ensure that $!foo died with a sensible error message when used outside the scope of mop.

So there's still quite a bit to be done. Expecting it in Perl 5.20 is unrealistic. Personally I think 5.22 would be unlikely to; 5.24 or 5.26 are more likely. Hopefully the current github implementation could then be refactored into a CPAN "MOP::Compat" distribution, providing a mostly compatible version for Perl 5.16+. (Similar to how MRO::Compat backports much of the Perl 5.10 module "mro" to Perl 5.6+.)

Reposting my comment from the original post lel...

Is it possible to have something like

package Cat;
class Food {
  has $!taste is ro;
  has $!brand is ro;
  has $!pounds is rw;
}

method Food::feed_lion { ... }
method Food::feed_cheetah { ... }

basically I want methods that I can define outside the class BLOCK (I dislike having one more level of indentation just for methods, and it is lazier to just s/^sub... /method .../ and wrap class around existing Moo* attribute declarations.

How does this destroy backwards compatibility? Code you wrote for Perl 5.8 will still work even if there's a MOP in 5.20 or 5.22.

That's not currently possible, but something like this should be:

package Cat;
 
sub _feed_lion { ... }
sub _feed_cheetah { ... }
 
class Food {
  has $!taste is ro;
  has $!brand is ro;
  has $!pounds is rw;
  method feed_lion { goto \&_feed_lion }
  method feed_cheetah { goto \&_feed_cheetah }
}

As chromatic said, no need to worry, the old object system will still be there. If you don't want to use the mop yourself, that is fine, because you will still be able to inherit from mop classes and mop classes will be able to inherit from your non-mop classes.

Zak - we very likely will support non-block classes as well (though perhaps not in the prototype) so it would be possible to do
class Food;
has $!taste is ro;
has $!brand is ro;
has $!pounds is rw;
method feed_lion { ... }
method feed_cheetah { ... }

Su-Shee - I plan to release the prototype to CPAN possibly before PPW (Oct. 3rd), mostly just need to write some docs for it. Of course it will still be highly experimental. As for which core release it will be in, I would love 5.22 but that might be ambitious as we still have to plan out the work to get it into core.

I was mostly professing my greed to have it sooner rather than later. :)

I like mop-redux and as one of "us common perl folks" and "regular users" I'd appreciate it a lot if we'd finally have SOMETHING released with core.

I'm just the vox populi nagging "are we there yet?".. :)

I am concerned by the loss of 'undef' as a value for attributes.
One of the benefits of having it is that it actually has
intrinsic meaning.

For example (pardon the pseudo-code) let's say that class Foo's constructor
accepts an optional object, bar.

In Moo+Types::Standard, one might do this

has 'bar' => (
is => 'ro',
isa => InstanceOf ['Bar'],
required => 0,
);

Let's posit that Bar's constructor returns undef upon failure. Old
school

Foo->new( bar => Bar->new );

will cause the validity check on bar to fail.

Will new school mop do the same? Or will it conclude that bar was
simply not specified because it was passed "undef", and continue?

The argument that currently undef means "not set" is not necessarily
the case. Here's a useful idiom, which allows for an optional
trailing argument whose value may be undef:

sub foo {

my ( $arg1, $arg2 ) = @_;

my $has_arg2 = @_ > 1;

}

undef can have a valid intrinsic meaning. It provides the ability to
specify a special state without having to define a special value (such
as NULL in C) which a) requires a common definition and acceptance by
the comminuity, and b) doesn't intersect the valid set of values for a
variable.

One of the aspects of Perl that I like is the unique position of undef
as a value. Take Lua, as a counter-example. It doesn't have a
separate sense of "existing but undefined", it has 'nil', which means
both non-existing and undefined, or whatever combination of those two
you prefer. There's no way to separate them out, and (in my
opinion/experience) leads to much less expressive code.


Diab - We do not treat undef as special, the decision was made to let undef work for has exactly as it works for my. Which is not how it works in Moo{se}, so in a sense we are returning the use of undef back to its natural perlish state, just as Larry intended.

Thank you for doing this.

My first impression: I mostly like how the code looks and that we get "safe" objects.
In a small example, the error messages I got were sensible and the script loaded fast.

I wonder if we could get the "colon-field notation" used by Object::InsideOut, e.g.
https://metacpan.org/module/Object::InsideOut#ATTRIBUTES

As for a more in-dept analysis it would be cool to read more posts of Perl gods regarding the mop.

true. I meant though, new code won't work on old perls. so If I write mop stuff in 5.20, then that's the minimum version. what i was suggesting is keeping the keywords the same as moose, then can always use Moose and it'll work in the old version.

I guess my problem boils down to I don't like all the new keywords etc. Like Moose, I love it, it's awsome. MooseX::Declare or Moops, not a fan, it makes my brain hurt because it's perl but mangled

nObody - it should be backportable to at least 5.16 and possibly even 5.14 (as a less performant CPAN version (basically the prototype with improvements and tweaks)).

As for the new syntax, actually it is not anything terribly new, its been part of Perl 6 for a good ~10 years now

;)

About Damien "dams" Krotkine

user-pic I blog about Perl.