Stevan Little

So, I mentioned in my recent talk that the p5-mop project is currently stalled but did not really expand on that topic much. In this blog post, I'm going to explain what has happened over the past year or so with the project, discuss some issues we ran into while trying to implement the p5-mop, and lay out why I think we need to do some work in directions other than the p5-mop for a while in order to get back the p5-mop back on track.

First let me start out saying that p5-mop is not dead, and that I would very much still like to finish implementing the "on CPAN" version of it. I believe Jesse "doy" Luehrs, my co-conspirator, feels this way as well. My outlook on taking that "on CPAN" version and merging it into the p5 core is currently somewhat less optimistic for reasons I will explain.

The Current Problems

The first problem with the p5-mop project is that, like all deep magic around extending the p5 core via modules, it is very mentally taxing and it takes a good day or two to re-wrap ones brain around it after leaving it, even for only a few days. This makes it very tricky to work on while still doing paying work at the same time (this is true of any really ambitious open source project, so this should come as no surpise). So while it is fortunate for Jesse and myself that our work lives have been busy lately, this does have the unfortunate side effect of neither Jesse nor I being able to pay enough attention to the p5-mop project.

That first problem is probably surmountable, but that is not the only one we have encountered. The second problem is actually just getting an implementation to work -- sanely -- in a "on CPAN" version, on top of Perl 5. Our original prototype used a lot of PadWalker and Scope::Guard magic to make attribute access work just like lexical variable access. Unfortunatly, our attempts stressed PadWalker and Scope::Guard, causing them to break in fabulously strange ways.

Our intention had always been to replace the PadWalker/Scope::Guard bits from the prototype with XS equivalents down the road, so when we ran into this problem initially, we tried to work around it by advancing the XS version instead. This ended up being pretty complex to actually implement, and is currently unfinished.

The prototype also depends on Package::Anon to handle method dispatching. This was also just a temporary hack in service of getting the prototype up and working. Unfortunately, in trying to make Package::Anon work and experimenting with how it "should" work, we ran into several limitations and bad assumptions within Perl's method dispatch. In particular the assumption that a class is a string which must be the name of a package, was very problematic. In the end, we were forced to conclude that Perl's method dispatch mechanism would almost certainly need to be changed to allow some kind of non-package based dispatching. This probably means some core level work, but we stopped short of digging into this deep enough to say for sure.

The Future Problems

Those are the two biggest issues we ran into (so far, anyway), but they are very complex and involved problems. Either one alone would be a significant hurdle, but they actually interrelate in fairly fundamental ways, making the overall complexity level of fixing them incredibly high. I do believe, given enough tuits, that they can be solved and a solid working prototype could be put on CPAN for people to start working with. But once on CPAN, we must ask the question, "What next"?

This is where I began pondering the Perl 5 core and what dropping a complex MOP into it might mean for the maintainers and even the codebase itself. One of the main goals behind adding p5-mop to the core was to try to unify the many different object implementations out there, but honestly it has been very difficult for both Jesse and myself to see a way forward that doesn't end up with a situation that is even more fragmented than what we have now.

These thoughts, along with a small detour into the sad state of asyncronous programming in Perl and a little bit of reading of the recent p5p "sub signatures" thread (and other related threads), really got me thinking a lot. It was all this which lead me to write my talk for OPW 2013. Not until I actually got to writing the slide about p5-mop did I realize that yes, it was stalled and that I did not currently know how to move it forward.

So, where does this leave us now?

In his talk about the future of Perl 5, Jesse Vincent laid out a plan for cleaning up the Perl 5 core in such a way as to maintain our backwards compatability commitments, but still be able to move the language forward. My experience with trying to implement the p5-mop on top of Perl 5 has now convinced me that achieving this goal of a cleaner, simpler core is critical for any sort of significant continued innovation or evolution of the Perl 5 languange.

This is why I am currently directing my efforts towards a prototype of what I envision an "Ultra Modern" Perl might look like. This will include a MOP very much like the p5-mop along with some other features I spoke about last year at DCBPW.

It will also (and perhaps more importantly) try to actually spec out what a "smaller p5 core" might look like from a language perspective. It is my belief that this admittedlly odd digression from the p5-mop project is actually neccessary to ensure the success of the project overall.

Am I right? Am I crazy? I guess only time will tell. In the meantime, if you'd like to help us actually try and figure out what an Ultra Modern Perl looks like, the MoeOrganization is over here and everyone is welcome to join.

(NOTE: Thanks to John "genehack" Anderson for his editing help on this article and much better title suggestion).