Code compilation alternative to Function()

Sean Silva silvas at purdue.edu
Tue Jan 21 15:50:44 PST 2014


On Mon, Jan 20, 2014 at 4:21 PM, Gustavs Tēbergs <fixplzsecrets at gmail.com>wrote:

> Following up with a benchmark.
>
> (I thought of this idea while working on a parser generator library,
> but unfortunately for my argument the library turned out really
> fast...)
>
> I decided to write a converter for Asm.js code to see roughly what
> happens when building code expression by expression, and it gives me
> existing programs to test with.
>
> https://gist.github.com/fixplz/8529003
>
> Results:
>
> * I took Asm.js code from https://github.com/kripken/ammo.js as a testing
> source
> * Tested with Node 0.10.17 Win7 64
> * The JS engine parses the source (1.6 MB) in 180 ms
> * The opcode reader traverses the corresponting opcode binary (1.8 MB)
> in 120 ms (the binary could be some 30% smaller by adding special case
> codes)
> * With conversion enabled the opcode reader takes 260 ms to convert back
> to JS
>
> The output concatenation is very fast. But it could be better.
>

Your `load` routine is not how I described and is inducing a ton of
cache-busting heap traffic. You need to put *all* strings in a single,
constant array and append references to those constant strings to a single
growing array. There should be no concatenations except for the final
.join(). I.e. something like:

var STRINGPOOL = ['do{', 'function', '(', ')', ...];
var KW_DO = 0;
var KW_FUNCTION = 1;
var LPAREN = 2;
var RPAREN = 3;
...
var outArr = [];

and then in order to "concatenate" a string to the output do something like:

outArr.push(STRINGPOOL[KW_DO]);

and then finally, after outArr contains the entire output, do:

return outArr.join('');

Really, what you need to do is to run a slight modification of your current
`load` routine *during compilation* and trace all strings that are
appended. Then uniquify all the needed strings and add them to a string
pool. Then you send down the wire:
1. The string pool.
2. A binary blob of indices into that string pool (or other "opcodes").

Then your "decoder" can be as simple as:
var stringPool = getStringPool();
var indices = getIndices();
var outArr = [];
for (var i = 0, e = indices.length; i !== e; ++i) {
    outArr.push(stringPool[indices[i]]);
}
return outArr.join('');

Really, this is a coding problem, and you just need to develop an efficient
codec for describing a string of JS. The approach in the Gist you linked is
not a very efficient codec. The codec used in the above simple loop is
probably not the most efficient, but if you add a couple special case
opcodes and/or use a cheap varint encoding for the indices (and
appropriately optimize the order of the string pool array to make good use
of it), then you can really get fast and small.

-- Sean Silva


>
> The conversion forgoes minimizing parentheses so the output is larger
> than the source (2.5 MB), so the total time of using a converter like
> this would be 260 ms to convert + 300 ms to parse.
>
> That means about 80% of the running time is taken up by conversion to
> JS here. If the conversion involved something more complex, like
> reading Ruby code, the proportion might decrease.
>
> I think my point stands - this performance is undesirable. It might be
> difficult to deploy web apps with dependencies on large foreign
> codebases, especially on mobile devices which likely take much longer
> to perform all of this.
>
> This is combined with the fact that use of Function() seems to be
> discouraged by the CSP proposal.
>
>
> On the other hand - I learned a bit about execution semantics in V8
> and found it effectively does what I wanted, only a little indirectly
> by building a string of JS syntax instead of IR. (So the problem
> severity in my mind is downgraded from mysterious action to
> "Function() kinda sucks".)
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20140121/7ae77429/attachment-0001.html>


More information about the es-discuss mailing list