Catch-all proposal based on proxies

Mike Samuel mikesamuel at gmail.com
Thu Dec 10 11:31:11 PST 2009


2009/12/10 Brendan Eich <brendan at mozilla.com>:
> On Dec 10, 2009, at 11:10 AM, Mike Samuel wrote:
>
>> 2009/12/10 Brendan Eich <brendan at mozilla.com>:
>>>
>>> On Dec 10, 2009, at 9:57 AM, Mike Samuel wrote:
>>>
>>>> Actually, I think if iterators are desired, proxies may be a good way.
>>>
>>> We have implemented iterators and generators since 2006 in JS1.7 (Firefox
>>> 2), and they do not involve proxies or property mutation including
>>> deletion.
>>> Those are costly features and I don't see why they are necessary for an
>>> iteration protocol.
>>>
>>> var examp1 = {a:1, b:2, c:3};
>>> var examp2 = [4, 5, 6];
>>>
>>> function keyIterator() {
>>>   let keys = Object.keys(this);
>>
>> This will fail for infinite iterators.
>
> So? That has nothing to do with the issue of using proxies for iterators.
> First, this example uses an object, and objects do not have infinitely many
> keys. Second, iterators as we've implemented them can compute next values
> without bound if you code them to do so -- the next function is free to do
> whatever it takes.

Sorry.  I should've paid more attention to the second example which is
an infinite iterator.

I was assuming iterators would work without clients explicitly
trapping STOP_ITERATION, which is what proxies would provide.   But I
suppose if we're doing proxies, we can also do the syntactic sugar to
do away with that in new looping constructs.

Proxy based iterators work well with existing loop constructs though
    while ('next' in iterator) doSomething(iterator.next);

This works because I used a different definition of iterator.  In my
example, the producer has almost exactly the same contract as the next
method of your iterator, differing only in the value thrown, and the
proxy served to convert it to an iterator.
The iterator then an object such that
   - there is no next property iff the iterator is exhausted !('next'
in iterator)
   - the next value in the series can be retrieved by reading the next
property which advances the iterator
   - optionally, setting or deleting the next property mutates the
last element returned on the underlying collection





> I'm not sure why you responded only to this line.
>
> /be
>
> (Wow, overcite :-P)
>
>
>>
>>>   let i = 0;
>>>   return {
>>>       next: function () {
>>>           if (i == keys.length)
>>>               throw StopIteration;
>>>           return keys[i++];
>>>       }
>>>   };
>>> }
>>>
>>> function indexIterator() {
>>>   let self = this;
>>>   let i = 0;
>>>   return {
>>>       next: function () {
>>>           if (i == self.length)
>>>               throw StopIteration;
>>>           return i++;
>>>       }
>>>   };
>>> }
>>>
>>> Object.defineIterator(examp1, keyIterator);
>>> Object.defineIterator(examp2, indexIterator);
>>>
>>> for (let i in examp1)
>>>   print(i + " (type " + typeof i + ")");
>>>
>>> for (let i in examp2)
>>>   print(i + " (type " + typeof i + ")");
>>>
>>> ---
>>>
>>> example output:
>>>
>>> a (type string)
>>> b (type string)
>>> c (type string)
>>> 0 (type number)
>>> 1 (type number)
>>> 2 (type number)
>>>
>>> ---
>>>
>>> Notes:
>>>
>>> 1. After Python, the meta-level handler is a function that returns an
>>> iterator for its receiver object. Yeah, that means |this| -- but it's
>>> just a
>>> convention. The handler could take an explicit obj parameter instead.
>>>
>>> 2. After Python, an iterator is an object with a next() method returning
>>> the
>>> next value in the iteration, or throwing StopIteration to end the
>>> iteration.
>>>
>>> 3. The StopIteration object is a well-known singleton (like Math). In our
>>> implementation each frame has one but any will do to terminate iteration
>>> (they all have the same [[Class]]).
>>>
>>> /be
>>>
>>> P.S. Here are some shims from JS1.[78] in SpiderMonkey to ES5 plus
>>> defineIterator:
>>>
>>> Object.defineProperty = function (obj, name, desc) {
>>>   return obj.__defineProperty__(name, desc.value,
>>>                                 !desc.configurable, !desc.writable,
>>> !desc.enumerable);
>>> };
>>>
>>> Object.defineIterator = function (obj, iter) {
>>>   Object.defineProperty(obj, '__iterator__', {value: iter});
>>> };
>>>
>>>> Code below shows a problem where having an object delete it's own
>>>> property means hard choices have to be made over when to throw an
>>>> exception.  Proxies are fundamentally lazier and so dodge this issue:
>>>>
>>>> var myIterator = iterator(function () {
>>>>  var i = 0;
>>>>  return function () {
>>>>  if (i === 10) throw STOP_ITERATION;
>>>>  return i++; };
>>>>  }());
>>>>
>>>> while ('next' in myIterator) { alert(myIterator.next); }
>>>>
>>>> var STOP_ITERATION = {};
>>>>
>>>> function iterator(producer) {
>>>>  var produced, pending;
>>>>  function fetch() {
>>>>  if (produced) { return; }
>>>>  try {
>>>>    pending = producer();
>>>>  } catch (e) {
>>>>    if (e !== STOP_ITERATION) { throw e; }
>>>>    return;
>>>>  }
>>>>  produced = true;
>>>>  }
>>>>  fetch();
>>>>  if (!produced) { return {}; }
>>>>  var it = {
>>>>  get next() {
>>>>    var result = pending;
>>>>    pending = void 0;
>>>>    produced = false;
>>>>    try {
>>>>      fetch(); // throws too early?
>>>>    } finally {
>>>>      if (!produced) { delete it.next; }  // throws if frozen losing
>>>> result
>>>>    }
>>>>    return result;
>>>>  }
>>>>  };
>>>>  return it;
>>>> }
>>>>
>>>>
>>>>
>>>> 2009/12/10 Mike Samuel <mikesamuel at gmail.com>:
>>>>>
>>>>> 2009/12/9 Mike Samuel <mikesamuel at gmail.com>:
>>>>>>
>>>>>> On the climbing the meta, I'd like to understand how this might
>>>>>> interact with other proposals.
>>>>>>
>>>>>> get - already can execute arbitrary code due to getters
>>>>>> set - already can execute arbitrary code due to setters
>>>>>> in - cannot for non host objects
>>>>>> delete - cannot for non host objects
>>>>>> enumerate - cannot for non host objects
>>>>>> hasOwnProperty - cannot for non host objects
>>>>>>
>>>>>> Incidentally, is Object.prototype.hasOwnProperty(myProxy)
>>>>>> O(myProxyHandler.keys().length) for proxies?  This seems bad since a
>>>>>> for (in) loop that filters out non-own properties would be O(n**2) on
>>>>>> top of the loop body.
>>>>>>
>>>>>> If a future version of ES includes some kind of generator/iterator
>>>>>> scheme, then "in" and "enumerate" cease to be proxy specific.  But
>>>>>> iterators could be implemented as proxies if instead of providing
>>>>>> (has, keys, enumerate) we provide (prop, keyProducer) where
>>>>>> prop(property) -> one of (undefined, OWN, INHERITED), and keyProducer
>>>>>> returns a function, that each time it's called returns a string
>>>>>> property name or undefined to signal no-more.  Of course, trying to
>>>>>> freeze an object that returns a key name multiple times is hard to
>>>>>> define, but returning an array has the same problem.
>>>>>>
>>>>>> If there is a lazy key mechanism then iterators can be implemented as
>>>>>> proxies
>>>>>>  function iterator(producer) {
>>>>>>    var pending, produced = false;
>>>>>>    function fetch() {
>>>>>>      if (produced) { return; }
>>>>>>      try {
>>>>>>        pending = producer(); produced = true;
>>>>>>      } catch (e) {
>>>>>>        if (e !== NO_MORE_ELEMENTS) { throw e; }
>>>>>>      }
>>>>>>    }
>>>>>>    return Proxy.create({
>>>>>>        get: function (property) {
>>>>>>          if ('next' === property) {
>>>>>>            fetch();
>>>>>>            var result = pending;
>>>>>>            pending, produced = void 0, false;
>>>>>>          }
>>>>>>        },
>>>>>>       has: function (property) {
>>>>>>         return 'next' === property && (fetch(), produced);
>>>>>>       },
>>>>>>       keyProducer: function () {
>>>>>>         return function () { return fetch(), produced ? 'next' : void
>>>>>> 0;
>>>>>>       }
>>>>>>     });
>>>>>>  }
>>>>>> This doesn't solve generators, since the pausing semantics of yield
>>>>>> can't be easily implemented on top of proxies.
>>>>>
>>>>> Actually, a getter that can delete its own next property is all you
>>>>> need for iterators.
>>>>>
>>>>> function iterator(producer) {
>>>>>  var pending, produced;
>>>>>  fetch();
>>>>>  if (!produced) { return {}; }
>>>>>  var it = {
>>>>>  get next() {
>>>>>    var result = pending;
>>>>>    try {
>>>>>      pending = producer();
>>>>>      produced = true;
>>>>>    } catch (e) {
>>>>>      if (e !== STOP_ITERATION) { throw e; }
>>>>>      delete it.next;
>>>>>    }
>>>>>    return result;
>>>>>  }
>>>>>  };
>>>>>  return it;
>>>>> }
>>>>>
>>>>> for (var it = iterator(x), item; 'next' in it;) {
>>>>>  item = it.next;
>>>>>  ...
>>>>> }
>>>>>
>>>>>
>>>>>>
>>>>>> 2009/12/9 Mark S. Miller <erights at google.com>:
>>>>>>>
>>>>>>> On Wed, Dec 9, 2009 at 11:02 AM, Brendan Eich <brendan at mozilla.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On Dec 7, 2009, at 4:11 PM, Tom Van Cutsem wrote:
>>>>>>>>
>>>>>>>> Dear all,
>>>>>>>>
>>>>>>>> Over the past few weeks, MarkM and myself have been working on a
>>>>>>>> proposal
>>>>>>>> for catch-alls for ES-Harmony based on proxies. I just uploaded a
>>>>>>>> strawman
>>>>>>>> proposal to the wiki:
>>>>>>>>
>>>>>>>> http://wiki.ecmascript.org/doku.php?id=strawman:proxies
>>>>>>>>
>>>>>>>> Hi Tom, great to see this proposal. I took the liberty of making a
>>>>>>>> few
>>>>>>>> small edits; hope they're ok. I like the stratification and the
>>>>>>>> ab-initio
>>>>>>>> nature of the design -- the last seems to me to be a crucial
>>>>>>>> improvement
>>>>>>>> over past proposals, which may help overcome the "climbing the meta
>>>>>>>> ladder"
>>>>>>>> objection.
>>>>>>>> Some initial comments, pruned to avoid restating others' comments:
>>>>>>>> 1. This proposal obligates the catch-all implementor to delegate to
>>>>>>>> any
>>>>>>>> prototype object in has and get, to include unshadowed prototype
>>>>>>>> properties
>>>>>>>> in enumerate, to shadow if p in receiver.[[Prototype]] in put, and
>>>>>>>> to
>>>>>>>> do
>>>>>>>> nothing for delete proxy.p if !proxy.hasOwnProperty(p).
>>>>>>>> In general, handler writers have to implement standard
>>>>>>>> prototype-based
>>>>>>>> delegation if it is desired. This is probably the right thing, but I
>>>>>>>> wonder
>>>>>>>> if you considered the alternative where prototype delegation is
>>>>>>>> handled "by
>>>>>>>> the spec" or "by the runtime" and the proxy is considered "flat"?
>>>>>>>
>>>>>>> We did think about it, but it seemed needlessly less flexible. If
>>>>>>> such
>>>>>>> flat-and-delegate handling is desired, an abstraction can be built on
>>>>>>> top of
>>>>>>> ours that emulates it as a convenience. The reverse emulation seems
>>>>>>> difficult at best.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> 2. The fix handler returning undefined instead of throwing
>>>>>>>> explicitly
>>>>>>>> to
>>>>>>>> reject a freeze, etc., attempt is a bit implicit. Falling off the
>>>>>>>> end
>>>>>>>> of the
>>>>>>>> function due to a forgetten or bungled return will do this. Ok,
>>>>>>>> let's
>>>>>>>> say
>>>>>>>> the programmer will test and fix the bug.
>>>>>>>> But more significant: could there be a useful default denoted by
>>>>>>>> returning
>>>>>>>> undefined or falling off the end of the fix function? An alternative
>>>>>>>> interpretation would be an empty frozen object. This has symmetry
>>>>>>>> with
>>>>>>>> undefined passed (or no actual argument supplied) to Object.create.
>>>>>>>> It's a
>>>>>>>> minor comment for sure.
>>>>>>>
>>>>>>> Since the undefined may be the result of a bug as you say, it seems
>>>>>>> worse
>>>>>>> for the bug to silently result in fixing the proxy into an empty
>>>>>>> frozen
>>>>>>> object. We think the current "noisy" behavior better supports
>>>>>>> defensive
>>>>>>> programming.
>>>>>>>
>>>>>>>>
>>>>>>>> 3. Mozilla's wrappers (proxies, membranes), which we pioneered for
>>>>>>>> security purposes (e.g. for DOM inspectors where privileged JS is
>>>>>>>> interacting with web content objects) and which have been copied in
>>>>>>>> other
>>>>>>>> browsers (at least WebKit), implement === by unwrapping, so two
>>>>>>>> wrappers for
>>>>>>>> the same object are === with that object, and with each other.
>>>>>>>
>>>>>>> In answer,
>>>>>>>
>>>>>>>
>>>>>>> <http://wiki.ecmascript.org/doku.php?id=strawman:proxies#an_identity-preserving_membrane>
>>>>>>> preserves === correspondence on each side of a membrane. Now that we
>>>>>>> have a
>>>>>>> concrete catchall proposal adequate to build membranes, we'd like to
>>>>>>> restart
>>>>>>> our discussions with Mozilla (JetPack, etc) about whether you could
>>>>>>> rebuild
>>>>>>> some of your C++ membranes in JS code using these primitives. We
>>>>>>> should
>>>>>>> follow up offlist.
>>>>>>>
>>>>>>>>
>>>>>>>> The proxies proposal does not have an unwrapped object, although
>>>>>>>> super? is
>>>>>>>> similar. Later in the proposal, you write "meta-level code will
>>>>>>>> ‘see’
>>>>>>>> the
>>>>>>>> proxy rather than the object it represents." This sounds more like
>>>>>>>> wrappers
>>>>>>>> as we use them -- there is always a wrapped object and its proxy or
>>>>>>>> wrapper.
>>>>>>>> The alternative of not trapping === is a leaky abstraction that
>>>>>>>> inevitably
>>>>>>>> breaks some programmers' expectations. Our early wrappers did not
>>>>>>>> hook
>>>>>>>> ===,
>>>>>>>> but eventually we settled on the unwrap-before-=== behavior based on
>>>>>>>> testing.
>>>>>>>>
>>>>>>>> This is a use-case I wanted to bring to your attention (Mike Samuel
>>>>>>>> raised
>>>>>>>> it in his reply by suggesting a Proxy.proxies predicate; his
>>>>>>>> [[Class]]
>>>>>>>> question also gets to the broader issue of transparency vs. leaky
>>>>>>>> proxy
>>>>>>>> abstractions). Our wrapper experience suggests allowing === to be
>>>>>>>> hooked in
>>>>>>>> a constrained way, for certain kinds of proxies. It could be that
>>>>>>>> this
>>>>>>>> use-case can't be served by a standardized, general proxy/catch-all
>>>>>>>> proposal, and must be done under the hood and outside of the ES
>>>>>>>> spec.
>>>>>>>
>>>>>>> To avoid some "climbing meta ladder" issues, we purposely distinguish
>>>>>>> between what we consider base-level operations, such as x.foo, and
>>>>>>> meta-level operations, such as Object.getOwnProperty(x, 'foo'). We
>>>>>>> attempt
>>>>>>> to be as fully transparent (leak free) as reasonably possible at
>>>>>>> virtualizing base level operations. We attempt to be fully
>>>>>>> non-transparent
>>>>>>> (leak like a firehose) to meta-level operations. Some of our
>>>>>>> classification
>>>>>>> may seem weird: Object.prototype.toString() is meta-level. It can be
>>>>>>> used to
>>>>>>> reveal that an object is a trapping proxy.
>>>>>>> Object.getOwnPropertyNames()
>>>>>>> is
>>>>>>> meta-level. Object.keys() is base level.
>>>>>>> The properties of === that we feel need to be preserved:
>>>>>>> 1) "x === y" does not cause any user code to run.
>>>>>>> 2) "x === y" neither gives x access to y nor vice versa.
>>>>>>> 3) "typeof x !== 'number' && x === y" mean that x is operationally
>>>>>>> identical
>>>>>>> to y in all observable ways. Substituting the value for x with the
>>>>>>> value of
>>>>>>> y cannot change the meaning of a computation.
>>>>>>> 4) "x === y" implies that the Grant Matcher
>>>>>>> <http://erights.org/elib/equality/grant-matcher/> may safely send the
>>>>>>> money
>>>>>>> to either x or y.
>>>>>>> A wrapper is not identical to the object it is wrapping, or there
>>>>>>> wouldn't
>>>>>>> be any point in wrapping it. Thus, they can't be ===.
>>>>>>> Two independent wrappers on the same object may behave differently,
>>>>>>> depending on their definers. Thus they can't be ===.
>>>>>>> Except for two proxies with identical parts, such as two object
>>>>>>> proxies
>>>>>>> with
>>>>>>> identical handlers and supers. However, as shown by our example
>>>>>>> membrane,
>>>>>>> one can just use an Ephemeron table to avoid creating semantically
>>>>>>> identical
>>>>>>> duplicate proxies, preserving === without magic.
>>>>>>>
>>>>>>>>
>>>>>>>> 4. The [[Get]] versus [[Invoke]] rationale: indeed performance is a
>>>>>>>> concern, but existing engines also specialize callee-computation
>>>>>>>> distinctly
>>>>>>>> from get-value, in order to optimize away Reference types. The ES
>>>>>>>> specs so
>>>>>>>> far do not, instead using the internal Reference type to delay
>>>>>>>> GetValue so
>>>>>>>> as to bind |this| to the Reference base when computing a callee and
>>>>>>>> its
>>>>>>>> receiver as part of evaluating a call expression.
>>>>>>>>  I think it is an open question whether a future spec, especially
>>>>>>>> one
>>>>>>>> using a definitional interpreter, will stick to References. If we
>>>>>>>> end
>>>>>>>> up
>>>>>>>> making the distinction that all practical implementations already
>>>>>>>> make,
>>>>>>>> between get-as-part-of-callee-computation and all other get-value
>>>>>>>> "gets",
>>>>>>>> then I don't think this rationale is so strong.
>>>>>>>> In general over-coupling to ES5 may not help either a new
>>>>>>>> Harmony-era
>>>>>>>> proposal to "get in", or to be as complete or expressive as it
>>>>>>>> should
>>>>>>>> be. So
>>>>>>>> a rationale based on choices or limitations of ES1-5 seems weak to
>>>>>>>> me.
>>>>>>>
>>>>>>>
>>>>>>> +100.
>>>>>>> I would love to see the concept of References disappear, and to see
>>>>>>> the
>>>>>>> ([[Get]], [[Call]]) pairs in the spec that really mean "call this as
>>>>>>> a
>>>>>>> method" be rewritten as [[Invoke]]s. In that case, I would
>>>>>>> enthusiastically
>>>>>>> agree that this catchall proposal should be upgraded with an invoke()
>>>>>>> trap.
>>>>>>> Note how this would make our membrane code simpler and more
>>>>>>> efficient.
>>>>>>> Rather than a get() trap at the choke point that creates and returns
>>>>>>> a
>>>>>>> function, we'd simply have an invoke() trap whose body is that
>>>>>>> function's
>>>>>>> body.
>>>>>>> I'd like to understand better how we could get rid of References.
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks again for this proposal,
>>>>>>>
>>>>>>> You're welcome. It was fun!
>>>>>>>
>>>>>>> --
>>>>>>>  Cheers,
>>>>>>>  --MarkM
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> es-discuss mailing list
>>>>>>> es-discuss at mozilla.org
>>>>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>>
>
>


More information about the es-discuss mailing list