Transpiling JavaScript with Marpa

In order to transpile JavaScript to (any language but first) perl(5), I wanted to have a generic methodology, independent of the target language. The proof of concept being the default transpiling of JavaScript to JavaScript.

Marpa::R2, came again to the rescue: Marpa allows the programmer to have introspection on the grammar. This mean that one can:

Let's be concrete: If, in the Marpa grammar, you specify an action in the userspace like that:

:default ::= action => valuesAndRuleId
lexeme default = action => [start,length,value]

The valuesAndRuleId method beeing:

sub valuesAndRuleId {
  my $self = shift;
  return {values => [ @_ ], ruleId => $Marpa::R2::Context::rule};
}

This mean that, in the parse tree value of the grammar:


  • every lexeme (let's say terminal) will have an associated value that is a reference to an array with three items:


    1. start: the position in the stream

    2. length: the length

    3. value: the terminal itself


  • every G1 (let's say non-terminal) rule will have an associated value that is a reference to hash with the following structure:


    1. ruleId: the nonn-terminal rule Id

    2. values: a reference to all RHS values


Let's evaluate the parse tree value of a simple JavaScript source code:

var i;

using the grammar in the __DATA__ section of this package. This will be:

$VAR1 = {
    'ruleId' => 230,
    'values' => [
    {
        'ruleId' => 227,
        'values' => [
        {
            'values' => [
            {
                'ruleId' => 233,
                'values' => [
                {
                    'values' => [
                    {
                        'values' => [
                        [
                         0,
                         3,
                         'var'
                        ],
                        {
                            'values' => [
                            {
                                'values' => [
                                [
                                 4,
                                 1,
                                 'i'
                                ],
                                {
                                    'values' => [],
                                    'ruleId' => 171
                                }
                                ],
                                'ruleId' => 168
                            }
                            ],
                            'ruleId' => 164
                        },
                        [
                         5,
                         1,
                         ';'
                        ]
                        ],
                        'ruleId' => 163
                    }
                    ],
                    'ruleId' => 146
                }
                ]
            }
            ],
            'ruleId' => 231
        }
        ]
    }
    ]
};

Now, using the generated transpilation package, one can reproduce a source code that would give the same AST as the original.

This is not only a proof of concept, but the starting point of JavaScript::Transpile, that should be a generic JavaScript transpilator, with perl5 as main target, providing also (yet another) a JavaScript running engine: the original source code will be transformed to an AST using MarpaX::Languages::ECMAScript::AST, then transpiled to perl.

Here is an example of full JavaScript to JavaScript of jquery-1.10.2, (the same here). This looks like jquery. But this is *not* a cut/paste of jquery. This is transpilation of JavaScript to JavaScript via an AST produced by Marpa.
Nothing prevents to transpile to another target language -;

2 Comments

A minor quibble: could we drop the term "transpiling"? There isn't an interesting distinction between compiling to machine code or a virtual machine versus compiling to a different high-level language. The interesting part happens in the AST; the source and target languages are just implementation details.

Otherwise, awesome work.

I think the cat may be out of the bag at this point. And personally I like to see this feline on the loose.

I had not heard the term until Jean-Damien used it., but Wikipedia has a good article on it:http://en.wikipedia.org/wiki/Source-to-source_compiler. I like the term very much and find it fills a true need. In describing the kind of work people are doing with Marpa, I've had trouble -- "compiler" suggests something that not only parses and produces another useable format, but also takes the original language down to the hardware level. I like to point out the great things being done with Marpa, but I also try to avoid exaggeration, and if you say "Marpa allows you to produce compilers easily and quickly", it suggests that Marpa makes it easy to write another GCC. I believe Marpa would make writing another GCC considerably easier, but to write a full compiler like GCC you have to solve a whole lot of problems that Marpa does not even begin to address.

The distinction between "compilers", which take the language a major distance down toward the metal, and "transpilers", which convert the languages at the same level of abstraction is extremely significant in theory and in practice. Having a word for it makes it easier to talk about it.

Leave a comment

About Jean-Damien Durand

user-pic About::Me::And::Perl