Iterators, generators, finally, and scarce resources (Was: April 10 2014 Meeting Notes)

[Kevin Smith]

I agree with pretty much everything Andy had to say, and would like to add
a meta-perspective:

We should be viewing this as a last-minute feature request.  The churn that
this request has introduced is (in my opinion) exactly the kind of problem
that the ES7 process is meant to address.  In fact, I would go so far as to
say that requirements churn has been the number one problem during ES6
development.

If the "iterator-as-resource-manager" feature is truly desirable (and I'm
not even convinced that it is), then the appropriate action is to figure
out a way to defer it until ES7.  It is simply too late to add this feature
now.



On Tue, Apr 29, 2014 at 3:40 AM, Andy Wingo <wingo at igalia.com> wrote:

> On Fri 25 Apr 2014 16:22, Domenic Denicola <domenic at domenicdenicola.com>
> writes:
>
> >> (2) not well-motivated according to some participants of the
> > discussion (e.g., it's not necessarily a good idea to rely on
> > finally-blocks for scarce resource management in the first place, since
> > they provide only weak guarantees either way).
> >
> > This to me is the most worrying point. I feel like Andy summed things up
> well in this old thread:
> >
> > http://esdiscuss.org/topic/yield-desugaring
> >
> > I do not feel that his arguments were ever rebutted.
>
> So in the meeting notes the consensus was to send the issue to
> "generator champions" -- brendan and david herman, and somehow I ended
> up on CC.  We had some backs and forths but as it seems that TC39
> members are choosing to discuss this issue here, I'll repost my initial
> note here.  Please read the meeting notes for a description of Jafar's
> use case.  I'm a bit grumpy that this is being brought up again, and
> this late, and in multiple forums, but as it seems that people want to
> talk about it again, that talking about it again is the thing to do...
>
>                             *  *  *
>
> If I may summarize Jafar's argument, it's that the iterator in a for-of
> may hold a scarce resource, like a file descriptor, and because of that,
> for-of should be able to release this scarce resource on an early exit
> via "break".  The provisional consensus elaborates a method to do this.
>
> Is this a fair summary?
>
> I sympathise with Jafar's plight but I think that the current setup is
> the best we can do.  The summary of my argument is this:
>
>   (1) calling return() on iterators is rarely appropriate;
>
>   (2) return() in generators is semantically weird; and
>
>   (3) making for-of call return() on early exit is expensive at run-time.
>
> I should note first that this situation is not limited to generators, so
> the starting point mention of "finally" blocks is something of a
> distraction.  To the extent that this issue applies to generators, it
> also applies to other kinds of iterators.  Indeed I expect that in
> practice most iterators in an ES6 program will be map, set, and array
> iterators, which in practice will not be implemented with generators.
> Incidentally I think that if TC39 decided to re-add this method, it
> should be called close() instead, because it doesn't make sense to
> "return" from a non-generator iterator.
>
>
> == Calling return() on iterators is rarely appropriate
>
> Again I do sympathise with the use case, but we should start with a
> discussion of what is the common case.
>
> If we knew that the @@iterator call in the for-of would return a fresh
> iterator, then it would make more sense to provide some means for
> closing on early exit.  Jafar argues that this is in fact the common
> case, which sounds about right to me.
>
> However, holding a scarce resource is also likely to be uncommon.  It
> certainly doesn't come up in the browser, for example.  I think it's
> reasonable in that rare case to require some thought on the part of the
> user as to what scarce resources they have acquired, and to arrange to
> release them as appropriate.
>
> Granted, if you are a user of an iterator, you might not know that it
> has a scarce resource.  So there are two cases here: one in which the
> iterator was created by its consumer, and one in which the consumer is
> decoupled from the producer.
>
> The first case is the one Jafar gives in his notes:
>
>   for (var line of openFile("foo.txt"))
>     if (line == '-- mark --')
>       break;
>
> However in this case it is possible to arrange to close the iterator,
> with a different interface:
>
>   var file = openFile("foo.txt");
>   try {
>     for (var line of lines(file))
>       if (line == '-- mark --')
>         break;
>   } finally {
>     file.close();
>   }
>
> Among other possibilities.  Something like Python's "with" might be
> appropriate here.  The point is that although in this case, calling
> return() on the iterator may indeed be appropriate, the desired behavior
> can still be implemented.
>
> Note that there is nothing special about for-of or iterators in this
> example; any abstraction that captures a scarce resource has to do the
> same thing.  It is not that generators are unable to abstract over IO --
> it is that they are unable to transparently abstract over scarce
> resource acquisition.  No surprise there.
>
> The other case is when you have an iterator consumer which is decoupled
> from the code that created the iterator, as in:
>
>   function (iterable) {
>     ...
>     for (var x of iterable) {
>       if foo(x) break;
>     }
>     ...
>   }
>
> But it is precisely in this case when you would *not* want to close the
> iterator, because you don't know its lifetime.
>
>
> == return() in generators is semantically weird
>
> I know the argument has already been made, but I would like to repeat my
> point (2) from
> https://mail.mozilla.org/pipermail/es-discuss/2013-May/030683.html,
> namely that close() "complicates the mental model of what happens when
> you yield."  It's really strange to consider a yield as not only an
> expression that produces a value, or possibly a point at which an
> exception could be thrown, but also a "return".  Bizarre.  It's a hazard
> to reading generator functions.
>
> Also, the insistence on a return() that doesn't run catch blocks seems
> to me to be ill-placed.  I think it's telling that the counter-examples
> are from Python, which has a different semantic model, as it has
> finalization.  Implementing abstractions over scarce resources in JS is
> going to necessarily involve different design patterns than those used
> by Python.  For the given use-case, throw() is entirely sufficient.  If
> you don't trust your generators to do the right thing on an exception,
> you shouldn't be acquiring scarce resources!
>
> Finally, the given use-case is incompletely specified; a loop can exit
> prematurely through exceptions as well as through "break".  So really
> what is proposed is a finally block in every for-of statement, which
> brings me to my next point...
>
>
> == Calling return() on early exit from for-of is expensive
>
> Wrapping a try/finally around each for-of is going to be really
> expensive in all engines right now.  I'm skeptical about our ability to
> optimize this one away.  Avoiding try/catch around for-of was one reason
> to move away from StopIteration, and it would be a pity to re-impose
> this cost on every for-of because of what is, in the end, an uncommon
> use case.  I think the expected result of doing this would be
> performance lore to recommend using other iteration syntaxen instead of
> for-of.
>
>
> There's no perfect answer when it comes to abstractions over scarce
> resources.  Given the constraints of what JS is, its finalization model,
> its deployment in the browser, and its engines, for me the status quo is
> the best we can do.  I know that for people that open file descriptors,
> that's somewhat unsatisfying, but perhaps such a cross is what goes with
> the crown of being a true Unix hacker ;)
>
> Regards,
>
> Andy
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20140429/dfa0dda8/attachment.html>