Catch-all proposal based on proxies

[Mike Samuel]

2009/12/10 Brendan Eich <brendan at mozilla.com>:
> On Dec 10, 2009, at 11:31 AM, Mike Samuel wrote:
>
>> I was assuming iterators would work without clients explicitly
>> trapping STOP_ITERATION, which is what proxies would provide.   But I
>> suppose if we're doing proxies, we can also do the syntactic sugar to
>> do away with that in new looping constructs.
>
> We do indeed have for-in loops, comprehensions, and generator expressions
> all automatically catching StopIteration in SpiderMonkey and Rhino. There is
> hardly ever a reason to write a try/catch -- just as in Python.
>
>
>> Proxy based iterators work well with existing loop constructs though
>>   while ('next' in iterator) doSomething(iterator.next);
>
> There are lots of convenient ways to express iteration, but one that
> mandates proxies is inherently heavier and harder to optimize than I think
> we (implementors or in some cases users) want.
>
>
>> This works because I used a different definition of iterator.  In my
>> example, the producer has almost exactly the same contract as the next
>> method of your iterator, differing only in the value thrown, and the
>> proxy served to convert it to an iterator.
>> The iterator then an object such that
>>  - there is no next property iff the iterator is exhausted !('next'
>> in iterator)
>>  - the next value in the series can be retrieved by reading the next
>> property which advances the iterator
>>  - optionally, setting or deleting the next property mutates the
>> last element returned on the underlying collection
>
> The last element, or the next element?

Previous as in java Iterators.


> Proxies can propagate effects to other objects, for sure, but this is not
> only expensive, it's also hard to analyze. An iteration protocol should be
> more functional (pure).



> We chose to borrow from Python first in the spirit of programming language
> design by borrowing from older languages, second to reuse developer
> brainprint.

Fair enough.


> But the functional (with light OO dusting on top, which could be removed as
> noted -- getting rid of |this|) flavor of the iteration protocol in
> SpiderMonkey and Rhino JS1.7+ is winning both for implementors and users, in
> our experience.



> To criticize our JS1.7/1.8 experience a bit:
>
> A. The obvious problem is that we hang the meta-level handler off the
> base-level object, as Python does with its __iter__. But
> Object.defineIterator seems like the fix for this bug.
>
> B. Creating an object with a next method instead of a closure may be one
> object too many. The object need not carry any state that's not in the next
> method's environment (as in the examples I showed).

> But the object does provide an identity separate from next, in which send,
> throw, and close methods are bound for generators. And the object allows
> state representation optimizations other than closure-based ones.

It would extend to set/delete as in java though.
Java has basically ignored set/delete on iterators in the syntactic
sugar around Iterables.


> C. Generators, being flat (one frame of activation saved) yet heap-escaping,
> don't always compose nicely. This has spawned PEP 380
> (http://www.python.org/dev/peps/pep-0380/).

Generators compose in some ways though the syntax can be awkward:
def zip(a, b):
  for a_el in a:
    yield (a_el, b.next())
  try:
    b.next()
  except StopIteration:
    pass
  except:
    raise AssertionError('%r not exhausted', b)

The part that does the work is straight-forward, and then the corner
case checking is, as usual, the most verbose.


> Generators are nevertheless quite convenient according to reports from our
> users, and pretty easy to implement in any implementation that compiles into
> bytecode or something more sophisticated, instead of walking ASTs to
> evaluate code.

Cool.


> It's important not to oversell generators as solving concurrency issues or
> being "coroutines" -- they are best thought of as the simplest way to write
> an iteration protocol handler (or more casually, "an iterator"). Here are
> the example iterators recast as generators:
>
> function keyIterator() {
>    let keys = Object.keys(this);
>    for (let i = 0; i < keys.length; i++)
>        yield keys[i];
> }
>
> function indexIterator() {
>    for (let i = 0; i < this.length; i++)
>        yield i;
> }
>
> When called, a generator returns an iterator, so these are factories. The
> functional version had to return objects with next methods:
>
> function keyIterator() {
>    let keys = Object.keys(this);
>    let i = 0;
>    return {
>        next: function () {
>            if (i == keys.length)
>                throw StopIteration;
>            return keys[i++];
>        }
>    };
> }
>
> function indexIterator() {
>    let self = this;
>    let i = 0;
>    return {
>        next: function () {
>            if (i == self.length)
>                throw StopIteration;
>            return i++;
>        }
>    };
> }
>
> This use-case is where generators shine.

I think your arguments against using proxies to implement iterators
are strong, so maybe keys/enumerate should be considered tentative so
that they can be informed by any separate work on iterators.

Again, separating out the concept of own-ness from key enumeration
might make that easier.


> /be
>