A reprieve for ES6 comprehensions

[Andy Wingo]

Hi,

dherman has argued that comprehensions should be removed from the ES6
draft:

  https://speakerdeck.com/dherman/a-better-future-for-comprehensions

The argument is that the existing comprehension mechanism is
incompatible with future extensions, notably parallelism.

I am not sure I buy this argument.  (Disclaimer: I did the work to adapt
SpiderMonkey's comprehensions to ES6 syntax/semantics, and have a
colleague that was working on doing the same in V8.  This does give me
some familiarity with the topic, but may blind me to alternatives.)

My reasoning is (1) that the difference between array comprehensions --
[for x of y z] -- and generator comprehensions -- (for x of y z) -- is
essential, not incidental.  (2) A unified syntax seems to me to be
incompatible with the ES6 external iterator design.  Finally, (3) the
current draft syntax both hangs very well with the rest of ES6, and it
is not incompatible with combinator-style extension.

1: Essential differences
==

Array comprehensions are eager.  Generator comprehensions are lazy.
Array comprehensions desugar into a do expression à la ES7.  Generator
comprehensions desugar into an IIGFE (immediately-invoked generator
function expression) -- sorta, anyway.  (Generator function desugaring
is not exact due to arguments/this/etc scoping and properties on the
generator function.)

This is an essential difference in the use of the iterator, and so it
makes sense to distinguish these at the source level as well.

2: Refactoring?
==

David brings up the argument that with parallel comprehensions, we then
would have three ways of iterating over a sequence, and that is reason
to see if we can refactor.  This is a good point.  However I don't see
it working out, because of (1).  The only way I could see it working is
if comprehensions became _internal iterators_: for example, the body of
the comprehension would be wrapped in an arrow function or something and
that function would get invoked in the different contexts.  This is
possible to optimize of course -- anything is possible -- but it is
going to really be a dog compared to what is currently in SpiderMonkey
(for example).  You are relying on the compiler to inline a whole stack
of functions.  There are many hazards here to getting good perf, and
we'd be baking that choice into the language.

3: Comprehensions match ES6
==

Comprehensions match ES6 very well as it is, without considering the
possible addition of parallelism.  They even match up well if we add the
other dimension of asynchronous comprehensions -- i.e. [for x of await y
z], (for x of await y z).  I don't think it's unreasonable to expect PJS
to also extend the syntax in this way -- we end up with 2³ squares in
our "matrix" -- eager vs lazy, sync vs async, parallel vs not -- and
that seems fine to me.

Finally the idea that iterators should have a combinator API like
y.if(pred).butNot(bar) or whatever -- that is totally a doable thing now
with the proposed addition of a common Iterator prototype, and it meshes
well both with for-of (which no one has proposed removing, and the same
arguments apply) and with comprehensions as they are in the draft.

In summary, I think Dave already did a good job with comprehensions and
they should stay in :)

Regards,

Andy