Questions/issues regarding generators

Allen Wirfs-Brock allen at wirfs-brock.com
Thu Mar 7 09:30:46 PST 2013


On Mar 7, 2013, at 7:37 AM, Andreas Rossberg wrote:

> We have started investigating the implementation of generators in V8,
> and a couple of questions popped up that are not quite clear from the
> proposal (and not yet in the draft spec, AFAICS):
> 
> 1) Are the methods of a generator object installed as frozen
> properties? (I hope so, otherwise it would be more difficult to
> aggressively optimise generators.)

We discussed the factoring of the generator objects at the Nov 27 meeting.  https://github.com/rwldrn/tc39-notes/tree/master/es6/2012-11 there is a sketch of the hierarchy we agreed to in in the notes https://dl.dropbox.com/u/3531958/tc39/generator-diagram-1.jpg 

TThe design I presented at the meeting is very close to that final one, answers this question, and is easier to read:  http://wiki.ecmascript.org/doku.php?id=meetings:proposed_generator_class_hierarcy_nov_2013.png 

Basically every generator function is a "constructor" object and has its own unique associated prototype that supplies the methods for its instances.  The "generator methods" aren't own properties of generator instances but instead are prototype properties.   Generator instances have whatever private state that is necessary to support the functioning of those inherited methods but otherwise have no own properties. 

I can see a plausible argument for saying all generator instances are created as frozen objects as this would prevent somebody dynamically inserting own over-rides of the prototype provided methods.


> 
> 2) Is yield* supposed to allow arguments that are not native generator objects?
> 
> 3) What happens if a generator function terminates with an exception?
> According to the proposal, nothing special. That implies that the
> generator is not closed. What happens when it is resumed afterwards?
> Moreover, is a StopIteration exception handled specially in this
> context?
> 
Good, questions.  I haven't worked through spec'ing those details yet.  Perhaps DHerman or Jason know.


> 4) Nit: can we perhaps rename the generator "send" method to "resume"?
> That is so much more intuitive and suggestive, Python precedence
> notwithstanding. :)
> 
> Apart from these questions, we also see a couple of issues with some
> aspects of the proposal. My apologies if the specific points below
> have already been made in earlier discussions (I could not find any
> mention).
> 
> 
> - The generator/iterable/iterator separation is somewhat incoherent.
> In particular, it makes no sense that it is a suitable implementation
> of an .iterator method to just return 'this', as it does for
> generators. The implicit contract of the .iterator method should be
> that it returns a _fresh_ iterator, otherwise many abstractions over
> iterables can't reliably work. As a simple example, consider:
> 
>  // zip : (iterable, iterable) -> iterable
>  function zip(iterable1, iterable2) {
>    let it1 = iterable1.iterator()
>    let it2 = iterable2.iterator()
>    let result = []
>    try {
>      while (true) result.push([it1.next(), it2.next()])
>    } catch(e) {
>      if (isStopIteration(e)) return result
>      throw e
>    }
>  }
> 
> You would expect that for any pair of iterables, zip creates an array
> that pairs the values of both. But is a generator object a proper
> iterable? No. It has an .iterator method alright, but it does not meet
> the aforementioned contract! Consider:
> 
>  let rangeAsArray = [1, 2, 3, 4]
>  let dup = zip(rangeAsArray, rangeAsArray)  // [[1,1], [2,2], [3,3], [4,4]]
> 
> and contrast with:
> 
>  function* enum(from, to) { for (let i = from; i <= to; ++i) yield i }
> 
>  let rangeAsGenerator = enum(1, 4)
>  let dup = zip(rangeAsGenerator, rangeAsGenerator)  // Oops!
> 
> Although a generator supposedly is an iterable, the second zip will
> fail to produce the desired result, and returns garbage instead.

I'm not sure I convinced by this.  An iterator instance represent a single specific iteration.  Your second example is really a user bug and should be coded as:
let dup = zip(enum(1,4), enum(1,4));

Zip's informal contract should state that if iterators are passed as arguments they need to be distinct objects. If you want to implement it defensively, you can add a  check for that pre-condition.

I can see where there might we some utility for a method that tests a iterator instance to determine whether it is in its initial state, or not.

I can also see utility in a "clone" method for some iterators (makes a new iterator over same data but in the initial state), but it the general case we can't assume that this is possible for every iterator.

> 
> The problem boils down to the question whether a generator function
> should return an iterable or an iterator. The current semantics
> (inherited from Python) tries to side-step the question by answering:
> "um, both". But as the example demonstrates, that is not a coherent
> answer.

I think it is pretty clear that the current semantics is that a generator function always returns an iterator.  The question is really about the additional protocol that is provided that allows collections and iterators to be used interchangeably. 

Perhaps it would be clearer if the @@iterator methods was named "asIterator", but that still wouldn't change the client bug in your example.

> 
> The only way to fix this seems to be the following: a call to a
> generator function should NOT return a generator object directly.
> Rather, it returns a simple iterable, whose iterator method then
> constructs an actual generator object -- and multiple calls construct
> multiple objects. In the common case of the for-of loop, VMs should
> have no problem optimising away the intermediate object. In the
> remaining cases, where the result of a generator function is used in a
> first-class manner, the object actually ensures the right semantics.
> 

Refer aback to the class hierarchy diagram.  This would make the overall iterator/generator object model even more complicated.  I don't wee enough benefit in doing so. 



> 
> - Finally, at the risk of annoying Brendan ;), I think we should
> (again) revisit the decision to use an exception to mark
> end-of-iteration. Besides the usual reservations and the problems
> already discussed in earlier threads, it has some rather ugly
> implications that I cannot remember being mentioned before:
...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20130307/b2c5c1fb/attachment.html>


More information about the es-discuss mailing list