Martin Berends

The community at #perl6 on freenode derived much pleasure from studying and discussing the entries to Masak's Perl 6 Coding Contest 2010. Five problems, five contestants, 26 entries (go figure) all used Rakudo. Since Niecza is the new Perl 6 kid on the block, it seemed an interesting idea to try it out on the submissions.

Alas, it was very disappointing to see error after error from Niecza. The results are included below for closer analysis. During testing it began to look as if Niecza would not run any entries, but fortunately it managed with p3-fox and p3-matthias. (They produced wrong answers, but hey, running code!)

The scripts that died with "Parse failed" never ran - the parser considered the source code to be not Perl 6 STD grammar compliant. There are lessons here - maybe Rakudo is too lenient in places, or maybe STD needs to be tweaked. It is also possible that the scripts could be patched to comply with STD, masak++ has already suggested a project to investigate.

The errors that mention exceptions occurred at run time - this is where Niecza most likely needs enhancement. Many of those jobs are major work, but a keen hacker may find some Low Hanging Fruit amongst them. This was an expected outcome of the experiment - to identify bits missing in Niecza that Perl 6 power users want.

The ones that ran, but wrongly - they will probably provide hours of debugging fun to Niecza newbies. Or the contestants, who have some explaining to do.

Statistically one can prove anything, and on those grounds the strong correlation between two percentages must be very significant. Niecza passes 1663 Perl 6 spectests at the time of writing, which is just a bit under 10% of what Rakudo manages. And Niecza ran a bit under 10% of the Coding Contest entries that Rakudo ran. The significance is just beginning to sink in...

The conclusion is that the clouds over these results do have silver linings. Niecza deserves admiration for what it can do, and more contributors to fix what it cannot do yet.

Results

• p1-colomon
  export tags NYI at line 5
  Traits, postconstraints, and shapes on variable declarators NYI at line 22
  Missing block (apparently gobbled by undeclared routine?) at line 35
  Undeclared name: 'Int' used at line 32
  Parse failed
• p1-fox
  Undeclared routine: 'max' used at line 15
  Unhandled exception: Check failed
• p1-matthias
  Unhandled exception: Unable to resolve method Numeric in class Sub at line 4
• p1-moritz
  In parameter declaration, typename 'Int' must be predeclared (or marked as declarative with :: prefix) at line 45
  Parse failed
• p1-util
  Unhandled exception: System.Exception: Unable to find lexical &hyperunary
• p2-colomon
  In parameter declaration, typename 'Complex' must be predeclared (or marked as declarative with :: prefix) at line 64
  Parse failed
• p2-fox
  Unable to find lexical &infix:
• p2-matthias
  Unhandled exception: Series op NYI
• p2-moritz
  Unable to resolve method ast in class Any
• p2-util
  Action method trait_mod:of not yet implemented at line 18
• p3-colomon
  In parameter declaration, typename 'Int' must be predeclared (or marked as declarative with :: prefix) at line 18
  Parse failed
• p3-fox
  *runs* but produces the wrong output
• p3-matthias
  *runs* but produces the wrong output
• p3-moritz
  Unhandled parameter twigil . at line 8
• p3-util
  In parameter declaration, typename 'Int' must be predeclared (or marked as declarative with :: prefix) at line 13
  Parse failed
• p4a-colomon
  Export tags NYI at line 5
  In parameter declaration, typename 'Numeric' must be predeclared (or marked as declarative with :: prefix) at line 26
  Parse failed
• p4b-colomon
  Export tags NYI at line 5
  In parameter declaration, typename 'Numeric' must be predeclared (or marked as declarative with :: prefix) at line 26
  Parse failed
• p4-fox
  Undeclared routines:
  'max' used at line 42
  'undefine' used at line 36
  Parse failed
• p4a-moritz
  Malformed block at line 39
  Undeclared routines:
  'all' used at line 24,29
  'any' used at line 7
  Parse failed
• p4b-moritz
  Malformed block at line 43
  Undeclared routines:
  'all' used at line 28,33
  'any' used at line 11
  Parse failed
• p4b-util
  Unhandled exception: Unable to resolve method end in class Array at line 11
• p5-colomon
  Traits, postconstraints, and shapes on variable declarators NYI at line 14
  In has declaration, typename 'Int' must be predeclared (or marked as declarative with :: prefix) at line 18
  Malformed has at line 18
  Parse failed
• p5-fox
  No value in mainline available for parameter $j at line 24
• p5-matthias
  Unable to resolve method push in class Hash at line 7
• p5-moritz
  Unable to resolve method ast in class Any
• p5-util
  Unable to resolve method keys in class Array at line 12