My take on Modern Perl
Szagab recently provoked me. On his question "I wonder how to teach "Modern" Perl? http://szabgab.com/how-to-teach-modern-perl.html" I anwered him by pointing to the qore manual.
Modern Perl might be an established term already. As radical modernist (opposed to a radical post-modernist) I still like to use my own vision of a modern perl. Not directly opposing chromatic's modern perl. Just a real modern perl, as an outsider would consider it. Which just happens to be the qore feature set. Everything I would have done to make perl modern would have been what David Nichols already did in qore. Plus channels for IPC.
Qore is basically a modern perl, a rewrite with a modern vision. One can take a perl script, and optionally enhance it with types and background and get a qore speed-up and compile-time safety, but obviously a compatibility problem.
- Optional strong typing (what I plan to do in the near future)
- Fast native threading, enabled automatically, TID 0 for signals.
- Thread-safe library and types
- Deadlock detection
- Multi-core support as in Go, with SMP thread scheduler, but no channels yet.
- Modern rewritten parser (he did it in C++)
- Modern exception handling, without string parsing of die messages
- Unified network event handling
What is different:
sub return types are C-style. perl has no style so far.
[return_type] sub function_name([[type] variable1, ...]) {
statements;
}
const are true constants, used by the compiler. Currently only Const::Fast comes close, Readonly and constant not at all.
The class and namespace keywords deviate from perl packages, that they are modern equivalent of "what to expect from them". No magic @ISA variable for inheritance e.g.
class options are [inherits [private|public]
method options are [static] [synchronized] [private]
So as I enview it, a "Modern Perl" would take the same step as "qore" did, plus channels. The existing feature set of chromatic's Modern::Perl is in my opinion the false way. Do not add half-features (e.g. Moose, Devel::Declare, types, ...) without native core support, and struggle then later. First the basics should be supported than the sugar can be done upon it. We don't have the basics but are already using the sugar, in an inefficient and hackish way. perl5 is IMHO not enough modern yet. We are using classes without efficient class and method support, we allow types but do not use it, we call a feature ithreads, which has nothing to do with threads. Marc Lehmann called it in his strong "Why so-called Perl threads should die" rant an inefficient fork. Ops and data are copied on thread init, not shared, no copy-on-grow, no thread-safe datastructures and library functions. No efficient SMP and modern IPC support.
"Modern" would oppose the current policy to allow everything outside core but hardly support anything inside. That's entirely "post-modern". "Modern" would be native green threads in core, native Class::Mop, native types, native argument handling in core. Handling it outside is nice, but post-modern.
The whole "strong typing" slope gets very slippery very quickly.
int sub sum(int @values) { ... }
Ah. Now I can't sum floats.
num sub sum(num @values) { ... }
What about objects that overload numerical addition? Now we need a parametric type to say "supports some operator"
Able[+] sub sum(Able[+] @values) { ... }
In general we care a lot more about the abilities than the actual types. "Duck typing" - code to interfaces, not implementations.
void sub hello(Animal $x) { say "An ", $x->kind, " goes ", $x->noise }
Now I can't implement a cow or a sheep or a tyrannosaurus rex without subclassing Animal. Perl should not require that. Anything that responds to a kind and a noise method should suffice.
void sub hello(Able[kind,noise] $x) { ... }
Now lets consider applying types deeply on functions that take functional arguments, such as a simple map() case:
List[B] sub map(B sub(A $x) $function, List[A] @values) { ... }
Already this is feeling a little inconvenient, as if I had a List[int], the type system would have to know that it's OK to use
num sub square(num $x) { $x * $x }
as the mapping function, because an int can be a num. But now this map list is going to yield a list of nums, where in practice we know this will yield a list of ints. I can no longer simply use
my @nums :List[int] = ...;
my @squares :List[int] = map(\&square, @nums);
So maybe we come up with a way to say that square() is really a function that takes some type that derives from Able and yields the same type
[T] sub square([T extends Able[*]] $x) { $x * $x }
Thus we'll know that squaring a list of integers yields a list of integers. We can even square a list of matrices or any other objects.
But what about, say, square roots? Some types are closed under square roots (complex numbers), some are partly closed under the dual space of yielding a result or raising an exception (real numbers), some are not closed.
Complex sub sqrt(Complex $z) { ... }
Maybe[Real] sub sqrt(Real $x) { ... }
Real sub sqrt(int $x) { ... }
Now we need type-based multiple dispatch to know what the result is. Indeed, what -is- the return type of
my $i :int = 1;
my $r :Real = 2.0;
my $c :Complex = 3 + 4i;
@result = map(\&sqrt, $i, $r, $c);
For my final nail, consider my CPS version of map(); CPS::Functional::kmap. Having to type-annotate this becomes
void sub kmap(Array[A] $values, void sub(A $value, void sub(B $result) $k) $kbody, void sub(List[B] @result) $k) { ... }
These end-results are perfectly doable, but they feel like suddenly a lot of work for what ought to have been quite a simple problem. Perl should hard things possible, but keep easy things easy. I don't feel like these examples are easy.
I don't feel that adding a strong parameter/return value type system to Perl is going to work. You can create a new language that is like Perl but has this additional feature, but the new language you'll have created will not be in form, or in spirit, Perl.
That's the whole issue with typing.
If you overload you have to obey the original types.
Typing is done to forbid overloading and other slow (tie) run-time magic and optimize the code at compile time.
You optimize to float or int but when you allow both or even undef or strings your are on the slow run-time checked path.
Dealing with dynamic types in e.g. for map or sort will not be done with typed functions or methods of course. Only some special functions/methods which will benefit. Perl is not a language to type everything. Only some special parts. Same as with Lisp.
Types are purely optional and may generate better and faster code.
Ah; I see. So it's static typing for the purposes of applying optimisations, not for checking program correctness?
In that case, why not have the runtime perform JIT-style optimisations as and when it requires, if it determines them to be necessary. Call counting, etc... can avoid a heavy overhead initially, so time is best spent on those locations that need it most.
Statically applying type constraints at development time feels like premature optimisation. Optimise when you determine you need it - and where, not before.
What a wonderfully documented language !
Reini Urban wrote :
> Optional strong typing (what I plan to do in the near future)
> Dealing with dynamic types in e.g. for map or sort will not be done with typed functions ... Same as with Lisp.
Are you saying it is possible to expect perl 5.1[future] to support optional types ? Very Interesting, if yes.
Static typing is for both, applying optimisations (smaller and faster) and checking program correctness (slower and safer).
I see no real jit advantage, as our ops are already optimized. Typed ops would bring some benefit, but we already have a lot of ops, and loose more in our way of stack handling (for XS). Our stack is on the heap so has slow far memory access in contrast to fast c-stack access. We also have to cleanup our "stack" (on the heap) ourself, in contrast to a c-stack. The current Jit only wins big on Windows with its inefficient dll handling. 20x faster was the optimal gain, but no gain at all on linux. And jitting involves a lot of additional memory allocation.
Jit-style type handling is an option though. Mainly for integer loops, method lookup and string hashes. But what we can dynamically do in assembler should better be pre-compiled in C and used in the perl compiler/optimizer pass. Should no big problem for the optimizer to detect those cases.
I like optional strong typing in subroutine parameters because it helps when you know to which class parameter belongs. Sometimes it is very hard to found it out if you are fixing a bug in someone's code. Of course, it can be done with comments, but making this a part of language will force to update declaration in case of changes and will allow optimizations. Also, sometimes type checking code is written manually or with some code from CPAN (Moose, Attribute::Signature).
I have yet to see any type-based optimisations for runtime performance that didn't get in the way of things like object class mocking for unit tests, to take a random example.
I like the dynamic nature of perl. I like knowing that anywhere I take an object, all I really care about is the set of methods it supports. That anywhere it wants a string or a number, anything that provides the relevant operator overloads, will also do just fine. This gives me exactly the flexibility and power that are the main reason I write in Perl in the first place, rather than e.g. Java or C#.
If you start providing ways to cut off that dynamic expression, then you start restricting the power of the things you can easily do in that language.
great stuff - really happy to see you like qore.
BTW qore already has in svn now some major new optimizations for integer operations (mostly applied to local variables) and also "pseudo-methods" which are pretty cool - these are class-type methods that can be executed on any value, whereas the "pseudo-class" depends on the value type - for example there are now methods like
bool ::empty() {}
*string ::firstKey() {}
int ::typeCode() {}
...
pseudo-classes are in a (relatively flat) hierarchy - all pseudo-classes inherit from ....
the second example above (and other hash pseudo-methods) shows the value of pseudo-methods - before this development, determining the first key in a hash or object (for example) was very expensive (you had to write something like my *string $firstKey = (keys )[0]; - meaning a list of all keys had to be created and then you try to take the first element).
I now use types heavily when programming in qore due to the parse-time checks they add and also the run-time optimizations. Also because Qore is designed to be embedded - types are useful in function/method signatures so that arguments and return types are clear (for this %require-prototypes is useful). Also it makes for clearer documentation - because Qore was designed actually for enterprise development. I have a doxygen framework for converting qore to C++-like code just so doxygen an parse it - here is an example of documentation produced from this if you are interested: http://qore.org/tmp/amris/index.html )
anyway let me just say again that it was great to read your post about perl and qore.
thanks,
David
oops - I should have previewed my post - it really looks awful it turns out -also of course there is no bool::empty() pseudo-method... anyway
Far better than duck typing (at least as implemented in most languages) is what happens using, eg templates in C++. Because C++ is compiling, it will expand out those types in all the functions where they are used - and check that there is a typesafe match with the object type being passed and the method being called.
"Interfaces" and "Roles" are almost as good, but they still require that the class in question implements the role as passed.
I did recently see a language where you specified what a role means, and then any class which implements a definition in line with that is considered to implement the role. Which I thought was pretty neat, duck typing done right (so long as the required signatures themselves are type safe!)
LeoNerd: You still haven't got it.
I will provide *optional* ways to cut off generic types.
The user or library author can decide by themselves that certain data will be typed. And get safety, speed and less size for it. There is no need everything must be generic, there should be a way to enable programmers to restrict their data.
You leave the type off, and then you can overload or tie or upgrade it to your wish.
In lisp this optional typing is done with "(the type what)" http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node106.html and ftype for functions.
Our declarations look better, because they look like static C or java declarations, but technically they do the same is the lisp optional declarations. The type-checker can ignore it, core needs a way to trust these declarations and the compiler can optimize it.
For the types article in the german Perl Magazine I collected some examples of modules which currently already use types.
Lexical variable types:
$ cd ~.cpan/build
$ ack '^\s*my \w'
And function types in every Moose module. There should be a way that moose authors get some more advantage of their typing efforts, not only slowness and safety.
Reini, could you please tell me what channels are in this context (the technical idea behind this concept)?
Qore uses lightweight web service protocols for communication - different from IPC because the 2 communicating parties don't have to be on the same system. One big drawback is that so far I don't have any way of serializing objects over these protocols. The protocols supported so far are XML-RPC, JSON-RPC, and, by far my favorite, proprietary YAML-RPC (because AFAIK there is no real standard for this yet).
In a single process, threads will typically use Queue objects for communication in Qore (http://qore.org/manual/qore.html#Queue_Class).