Reini Urban

Now that p5p removed the broken lexical topic feature in 5.24 it's time to look back how that happened and why it should not have happened, and how cperl fixed that problem.

In 2013 Rafael Garcia-Suarez posted to p5p a plea to Salvaging lexical $_ from deprecation arguing that the recent deprecation of my $_ is wrong, and can easily be fixed.

He explained the problems pretty nicely on the language level, but forgot to explain the internal advantages of a lexical $_ over a global or localized $_. To add that it's pretty simple: A lexical $_ is an indexed slot on the pad array, calculated at compile-time, whilst a local or global $_ is a hash entry in the namespace. It's much faster to lookup, write and restore. The slot index is compiled and carried around in the op. That's why it's called lexical, it is resolved at compile-time, and not at run-time.

The best technical reason why a lexical $_ is needed is given by davem, the only smart guy in the room here at RT #94682. given uses an implicit my $_ internally, and every match uses it.

If you try to read the answers by p5p you will get headaches as their explanations are not only clueless and wrong, they even argue that using $_ lexically is semantically wrong! Eg. rjbs: "I think it is half-baked, confused, and confusing. I don't see how it can be made useful or straightforward, at the language level."p5p

He clearly is confused because of the tricky bugs on the ops he is talking about, but here he was arguing that the semantics are broken(!). Using $_ explicitly or implicitly; or global, local or lexical makes no difference, only to RJBS and to some other porters. Doy for example argues that my $_ should be removed because this example is broken:

my $_ = 'foo'; print = any { $_ eq 'a' } qw(a b c)

which I fixed in 10 minutes by using the lexical $_ in the XS implementation and not the global. See the simple patch at RT #113939

The underlying problems with the implementation of my $_ were quite trivial to fix. I just had to fix the initial check for OPp_TARGLEX in the cperl commit 597e929c9 August 2015, which I published quite well. But the message obviously didn't arrive at the p5p team, because the decided that my $_ cannot be fixed and needs to be removed end of August. This time without any deprecation period.

One other interesting new answer was by zefram that this bug if not fixable, because we'd need support for lexical $_ in every op which supports implicit $_ or takes blocks, such as grep and map, obviously not knowing the codebase that this is exactly what we were doing quite efficiently since 2001. There is e.g. a special lexical grep bit. A git grep GREP_LEX would have helped, which was added 2004 as a special bit to grep.

I think there's no way forward that looks at all similar to what we have. Any lexical topic variable would at least have to be selected on a per-construct basis: you can't have "my $_" causing grep and the like to use a lexical topic. A spearate "grep_lexical" operator should be fine, but if we go that route we should consider forcing the lexical topic variable to have a name other than "$_" to avoid the confusing shadowing. Actual "my $_" should be no more valid than "my @_". - June 2015.

There was no single answer to that clear message. In retrospect I would also don't know what to answer to so much nonsense in one single paragraph.

I concur with rgs that "In my opinion, the more important problem here is the impression that P5P is throwing away without much thought of a perfectly nice and modern language feature (for some value of modern that means "post-FORTRAN"). This could give the impression of a the lack of vision for Perl 5 (and reinforce the "perl is dead" death spiral as perceived by the outside world -- the Perl users)."

Well said, and I knew it beforehand before it became true. But it's pretty easy to foresee such desaster looking at the track record this team is piling up, and how they react to critism, reviews and help. Their latest attempt to come up with an unwritten religious CoC rule, that you need to have faith in the maintainers in order to talk to them is the final sign of "Death by Code of Conduct". It's pretty hard to have faith in someone who decides 90% on the wrong side and only by luck sometimes makes a right decision. Initially I thought this code of conduct could finally stop all the silly bullying and name calling, but it got even worse, and whenever I complain about such abuse p5p punishes the one who complains about abuse, which is clearly only in the interest of the p5p buddies to maintain their powers they are abusing, but not in the interest of anybody else.

Now that p5p decided on this schism and went forward with more totally wrong and outragious decisions to do more and more harm on the perl5 language, I can only recommend to use cperl, the perl5 implementation where such grave mistakes do not happen, where perl5 development actually continues in the spirit before p5p took it over, and where the p5p principles of ruling by incompetence, power and abuse are not tolerated. cperl decisions are made rationally, professionally and most of all "cperl is not a religion". You are allowed to show distrust and criticsm. And in cperl such bugs are actually getting fixed, and the language is not harmed.

You will also hopefully not see those incredibly silly mailinglist threads as the cited one above. It's atrocious. We can only hope that p5p is put to an end soon, and a proper development process can start. There's no way this can go forward like this.

Maybe I should finally mention that fixing the removal of the lexical-topic is not only done by skipping the removal of it. The new CX code e.g. assumes that all the $_ there are global, that no lexical $_ can exist, and fixing this already cost me a few hours when merging with 5.24.0. It's pretty hard to deal with artificially stupidified code. At least the new 5.24 CX code is so wonderful that's it's worth it. +20%