Immediate closing of iterators

Chris Hansen renselp at gmail.com
Fri Dec 22 09:05:44 PST 2006


> Automating close only for generators iterated by for-in loops leaves
> non-loop use cases that nevertheless need to clean up after
> themselves out in the cold.

I wouldn't call the absence of automatic closure "leaving them out in
the cold".  The coroutine generators described in PEP 342 are similar
to threads, and the analogous situation for threads would be to
guarantee that a thread is automatically "closed" and has all its
pending finally clauses run, even if it never returns.  If a thread
blocks on some call and never runs again I wouldn't consider it "being
left out in the cold" if the underlying system didn't forcefully close
it eventually and run finally clauses.  I would argue that the same
reasoning applies to generators: if you create them and want to make
sure they're closed then you have to close them yourself.  I don't
think people (except pythonians maybe) would consider that
unreasonable.

> But consider the general case, outside of the cramped world of web
> page scripts: XUL apps and extensions, or other SpiderMonkey
> embeddings, will use generators in various ways, and not be subject
> to DOS attacks.  These embeddings expose stateful APIs to trusted
> code, and failing to call close may leak an OS resource or fail to
> synchronize important state.

Since finalization is tied to the GC (which might be conservative) it
cannot, by definition, be relied on to release resources in something
that resembles a timely fashion, or at all.

> However, the try/finally issue still makes us want to follow Python,
> for general purposes (not just browser purposes).  Lacking a DOS
> threat, and assuming the generator is written correctly, we convinced
> ourselves this summer that finally clauses should run from close
> after the last yield in a try has returned a value, and the caller or
> the GC is done with the generator.  If the caller forgets to close,
> finally still should run (says the ES3 spec in all of the cases it
> defines).

I would claim that as soon as generators start to resemble threads or
coroutines the situation changes.  In java, if a thread blocks
indefinitely in a try clause then then the finally clause will never
be executed (at least not as far as I can tell).

> We say if you're bad, you still get finally
> guarantees. :-)

If you do have finally guarantees I think that's the right thing to
do.  Of course I'm arguing for the polar opposite: even if you're good
you don't get any guarantees :-).

There's another issue with finally guarantees that I don't think has
been mentioned: which thread is it that closes the generators?
Consider this example:

let genCount = 0;
function makeGenerator() {
  genCount++;
  return myGenerator();
}

function myGenerator() {
  try {
    yield 1;
    yield 2;
  } finally {
    genCount--;
  }
}

Imagine that the program has been running for a while and the GC
decides to set in right after makeGenerator has read genCount but
before it has incremented it.  If the GC happens to close some of
these generators then genCount will be in an inconsistent state when
execution continues.  If the GC doesn't then who does?

Sorry about the mail address mixup by the way; that's what you get
when you mix private and work mail...  I did think it was odd, though,
that the posting even got through since my work mail address isn't a
subscriber to es4-discuss.


-- Chris



More information about the Es4-discuss mailing list