You can see an example that uses a canvas to draw a spaceship at https://perl6.github.io/6pad/#5a889cc9d6385853af2ff6a3fa165662
6pad has been used in the Perl 6 advent calendar demonstrating that it's usable by community members not directly working on rakudo.js
Rakudo.js has been released on NPM as rakudo
The Rakudo.js backend has been fully merged into the master branches of rakudo and NQP
Look at the Rakudo NPM page for information on how to use it.
The NPM package release process is fairly complex. But a repo yarn workspace helps set everything up.
For deploying Perl 6 in the browser we have a Parcel plugin parcel-plugin-nqp.
The plugin will be upgraded to Parcel 2 once that has a non-alpha relase (or maybe even sooner).
You can get a prebuilt rakudo.js from npm with a single command
npm install rakudo
I have decided not to upload the rakudo.js tarballs to CPAN (while easily doable if anybody actually want to get it from there it seems nobody would benefit from that as it seems getting it from npm will be just more convenient for everyone).
We pass our chosen subset of roast tests in headless Chrome using puppeteer
We have a repo with a test runner that bundles up the tests using the parcel bundler and passes them to Chrome
A couple of tests don't work due to either being impossible in the browser (due to using file system IO) or rakudo itself having a fair amount of bugs that pop up when precompiling code (in usual runs tests aren't precompiled so they go undetected).
Those bugs are present in rakudo-moar too but because it can't currently precompile scripts only in code used in modules. I plan to devote some time to investigate how to make the roast test suit catch them after this grant.
See blog post (http://blogs.perl.org/users/pawel_murias/2019/04/rakudojs-update---running-tests-in-a-real-chrome.htm) for a more in depth explanation
In total that makes ~61% of total roast tests (the grant was targeting only a subset of the whole roast), which in practice allows the usage of a huge chunk of Perl 6 (with the biggest omission being multi threading and IO things).
As a bonus feature rakudo.js has some basic interopability with JavaScript.
This allows us to import js modules as well as call methods and get attributes of JS objects.
In the current form that allows use to use the DOM in the browser or when run in the browser load node.js modules like use chalk:from<node.js>
We have a Dart Pad inspired REPL called 6Pad that allows you to play with Rakudo.js without installing it
Rakudo.js has been released on NPM as rakudo
You can get a prebuilt rakudo.js from npm with a single command
npm install rakudo
I have decided not to upload the rakudo.js tarballs to CPAN (while easily doable if anybody actually want to get it from there it seems nobody would benefit from that as it seems getting it from npm will be just more convenient for everyone).
# We pass our chosen subset of roast tests in headless Chrome using puppeteer
We have a repo with a test runner that bundles up the tests using the parcel bundler and passes them to Chrome
A couple of tests don't work due to either being impossible in the browser (due to using file system IO) or rakudo itself having a fair amount of bugs that pop up when precompiling code (in usual runs tests aren't precompiled so they go undetected)
See blog post (https://blogs.perl.org/users/pawel_murias/2019/04/rakudojs-update---running-tests-in-a-real-chrome.html) for a more in depth explanation
In total that makes FILL-IN-PERCENTAGE of total roast tests (the grant was targeting only a subset of the whole roast), which in practice allows the usage of a huge chunk of Perl 6 (with the biggest omission being multi threading and IO things).
# You can get a prebuilt rakudo.js from npm with a single command (assuming you have the current version of node.js setup)
npm install rakudo
JavaScript interop
As a bonus feature rakudo.js has some basic interopability with JavaScript.
This allows us to import js modules as well as call methods and get attributes of JS objects.
Our test runner is in https://github.com/pmurias/perl6-js-roast-test-runner
It runs tests using a precompiled working version of rakudo.js fetch from npm using a roast revision we know passes.
In the browser we precompile tests before running which means the tests are run differently then
how they are tested on other backends.
As a result a whole bunch of tests fail under precompilation even on the Moar backend.
They need to be fixed as they are real bugs in Rakudo on all backends (as most proper codes
tends to live in precompiled modules not scripts) but that's work separate from the js backend.
The browser itself doesn't currently support directly writing to the users file system.
For that reason test cases that creates temporary files or directories are fudged.
Running external shell commands is not supported either.
However running code snippets in a fresh rakudo with is_run from Test::Util is emulated.
Tests for MAIN don't pass in the browser either as they don't really make sense.
Puppeteer uses a weird canary chrome version that has some issues with Array.shift
From now one that we have enough raw test passing ability the focus will be on making rakudo.js more usable for users.
Making 6pad a much cooler toy is also planned in the mid term plans.
Things as better examples and improved js interop (so that things such as WebGL/canvas/React.js can be used more easily) will make playing with 6pad (and the rakudo.js in general) a lot more fun.
A lot of the remaining tests depend on IO (and a such will only pass in the browser) or on different threading model than JS already has.
The IO ones will be addressed when needed but it's not a priority unless someone needs that when running on node.js.
The way the process works is that the Perl 6 test files get compiled to JavaScript and bundled by the parcel with everything they needs to run. The bundling includes the setting, runtime and even the whole Rakudo compiler (tests use EVAL a lot).
As as side node it turns out that for debugging purposes node bundled-everything.js emulates running in the browser very closely.
The first big obstacle was that a whole bunch of tests run a new rakudo process to execute some code.
In the browser running shell commands obviously doesn't work.
In the tests it's almost always done by using a &is_run from Test::Helper.
I added a new nqp::p6fakerun op that's called by that sub that runs rakudo in a new global context and hijacks the standard input, output and error. This required actually having a savable and restorable the global state (After refactoring I now keep most of it in a single global object). This took a ton of fiddling and changing stuff all over but hopefully will be also useable to precompile things in the same process to save on loading time. Some of the tests needed a few things faked (like a new faked pid or passing env variables) so I implemented that too.
The second big obstacle was the other backend don't precompile tests when running them (and neither does rakudo.js on node). It turns out that the backend indepenent part of rakudo has a bunch of undiscovered bugs. After finding this the hard way by wasting a lot of time on trying to find the root source of one or two of them in rakudo.js itself. I wrote a helper to run tests by wrapping them in modules to discover those. That approach runs into problems on some tests that can't be so easily wrapped so I'm currently working on way to precompile the tests directly on the moar backend. Once that's done I will add a spectest variant to spot those bugs easily and prevent more from getting introduced as well as help fixing all of the existing ones.
The third obstacle was that that node.js implements couroutines using the fiber module (which as far as I'm aware uses threads underneath). In the browser we use async/await for couroutines. I generate code that's usable by both approaches by emitting commented out /*async*/ and /*await*/ keywords and doing a fairly cheap textual substitution when needed later on.
It turned out a bunch of awaits (and a few asyncs) where missing which resulted in often strange bugs. I painstakingly found where they where missing and added them.
Some of the sequences tests found subtle bugs in our coroutines implementation.
The way our coroutine implementation handled exception was incorrect and needed to be fixed. I got rid of the currently handled exception stack too (and put it on the caller context).
For fixing async/await problems later found that running on a special "canary" version of node helps a lot as it has a new flag to print out stack traces for async/await.
The problem I'm going to work on next is getting source maps to work in the browser.
Some adhoc bugs caused by the bundling process will likely need to get fixed on the way.
While the underlying JavaScript VM is the same, there are some still some browser idiosyncrasies to solve.
As an example I'm currently I'm working on making &is_run work. A lot of tests use it to start up a new rakudo process, possibly passing some arguments and data on standard input and intercept it's output. Obviously in the browser we can't start a new rakudo and it needs to be faked.
In other news 6pad now supports basic DOM access (https://perl6.github.io/6pad/#4ee170f8043efe4802c8e6fb55d6a187), I'm planning some exciting stuff for 6pad but that's for the next update.
In other good news 6pad is up (https://perl6.github.io/6pad/).
6pad allows you to compile and run Perl 6 code from the comforts of your own browser.
For now Rakudo.js depends on BigInt support (which mean you should likely use 6pad with Chrome).
BigInt support is being worked on Firefox and all the Chrome derivatives should update their V8s soon.
6pad is still being worked on so it still likely has some bug.
Support for using React to create DOM elements is planned and should come soon.
6pad has support for running your gists so if you write your own spiffy Perl 6 example I'll show it to 6pad users.
Focus of the JS backend will now be on on polishing 6pad and fixing some failing tests.
It's not super usable and this moment and will likely explode if you try to have too much fun.
Hopefully this example will be replaced by something much more awesome soon.
Parcel.js is a web application bundler so it will take care of combining the JavaScript code Rakudo.js spits out with everything else your app needs to run.
Why Parcel.js instead of Webpack?
Mostly because Parcel while seemingly more buggy doesn't insist that hard on re-parsing the generated code.
I had a webpack plugin working for NQP at some point so it definitely is doable so webpack afficionados are welcome to contribute a plugin or entice me to write one.
Currently I'm using some cutting edge JS features (like BigInt) so I'm focusing on supporting Chrome.
I plan to spend some time on supporting other common browsers when Rakudo.js is more mature but luckily the evergreen browsers are constantly updating themselves and implementing new stuff so hopefully the problem will solve itself.
Truffle which is what we are using is a language implementation frameworks that creates an efficient JIT from a (sufficiently annotated) AST interpreter written in Java.
The implementation lives in the 'truffle' (https://github.com/perl6/nqp/tree/truffle) branch of the nqp repo.
nqp-truffle run the first NQP test today and I expect to get many more to pass in the coming days.
Rakudo.js has been in bugfixing mode recently.
Rakudo.js now uses NFG (Normal Grapheme Form) semantics in some places.
This means some string operations treat strings as sequences of graphemes instead of unicode code points. Graphemes are "user-perceived characters" (See http://unicode.org/reports/tr29/). This isn't done everywhere yet but it allows us to pass a bunch of roast tests.
Because JavaScript doesn't use graphemes underneath in it's string implementation like MoarVM does using NFG semantics can be much more expensive.
As such in low level setting code we often want to use the native javascript semantics when they are good enough.
To make that a choice I added a bunch of NFG aware op variants like (nqp::charsnfg) so we can pay the price only when it's necessary.
I have also implemented the Unicode Collation Algorithm (http://unicode.org/reports/tr10/) which allows us to sort strings in a unicode aware manner.
Javascript has a Intl.Collator functionality buitin (See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Collator) but unfortunately it uses a slightly different semantics so I had to write my own implementation.
Once documented and optimized it should be usable independently of the rakudo.js project (https://www.npmjs.com/package/unicode-collation-algorithm).
I have finally figured a tricky long standing bug with roles which means they now work.
A bunch of other less notable stuff has been fixed.
Now roughly 94% of the roast subset specified in grant deliverables pass so I now plan to focus on the remaining bugs and then move on to performance and usability.