Catch-all proposal based on proxies

Brendan Eich brendan at mozilla.com
Thu Dec 10 11:05:57 PST 2009


On Dec 10, 2009, at 9:57 AM, Mike Samuel wrote:

> Actually, I think if iterators are desired, proxies may be a good way.

We have implemented iterators and generators since 2006 in JS1.7  
(Firefox 2), and they do not involve proxies or property mutation  
including deletion. Those are costly features and I don't see why they  
are necessary for an iteration protocol.

var examp1 = {a:1, b:2, c:3};
var examp2 = [4, 5, 6];

function keyIterator() {
     let keys = Object.keys(this);
     let i = 0;
     return {
         next: function () {
             if (i == keys.length)
                 throw StopIteration;
             return keys[i++];
         }
     };
}

function indexIterator() {
     let self = this;
     let i = 0;
     return {
         next: function () {
             if (i == self.length)
                 throw StopIteration;
             return i++;
         }
     };
}

Object.defineIterator(examp1, keyIterator);
Object.defineIterator(examp2, indexIterator);

for (let i in examp1)
     print(i + " (type " + typeof i + ")");

for (let i in examp2)
     print(i + " (type " + typeof i + ")");

---

example output:

a (type string)
b (type string)
c (type string)
0 (type number)
1 (type number)
2 (type number)

---

Notes:

1. After Python, the meta-level handler is a function that returns an  
iterator for its receiver object. Yeah, that means |this| -- but it's  
just a convention. The handler could take an explicit obj parameter  
instead.

2. After Python, an iterator is an object with a next() method  
returning the next value in the iteration, or throwing StopIteration  
to end the iteration.

3. The StopIteration object is a well-known singleton (like Math). In  
our implementation each frame has one but any will do to terminate  
iteration (they all have the same [[Class]]).

/be

P.S. Here are some shims from JS1.[78] in SpiderMonkey to ES5 plus  
defineIterator:

Object.defineProperty = function (obj, name, desc) {
     return obj.__defineProperty__(name, desc.value,
                                   !desc.configurable, ! 
desc.writable, !desc.enumerable);
};

Object.defineIterator = function (obj, iter) {
     Object.defineProperty(obj, '__iterator__', {value: iter});
};

> Code below shows a problem where having an object delete it's own
> property means hard choices have to be made over when to throw an
> exception.  Proxies are fundamentally lazier and so dodge this issue:
>
> var myIterator = iterator(function () {
>  var i = 0;
>  return function () {
>    if (i === 10) throw STOP_ITERATION;
>    return i++; };
>  }());
>
> while ('next' in myIterator) { alert(myIterator.next); }
>
> var STOP_ITERATION = {};
>
> function iterator(producer) {
>  var produced, pending;
>  function fetch() {
>    if (produced) { return; }
>    try {
>      pending = producer();
>    } catch (e) {
>      if (e !== STOP_ITERATION) { throw e; }
>      return;
>    }
>    produced = true;
>  }
>  fetch();
>  if (!produced) { return {}; }
>  var it = {
>    get next() {
>      var result = pending;
>      pending = void 0;
>      produced = false;
>      try {
>        fetch(); // throws too early?
>      } finally {
>        if (!produced) { delete it.next; }  // throws if frozen  
> losing result
>      }
>      return result;
>    }
>  };
>  return it;
> }
>
>
>
> 2009/12/10 Mike Samuel <mikesamuel at gmail.com>:
>> 2009/12/9 Mike Samuel <mikesamuel at gmail.com>:
>>> On the climbing the meta, I'd like to understand how this might
>>> interact with other proposals.
>>>
>>> get - already can execute arbitrary code due to getters
>>> set - already can execute arbitrary code due to setters
>>> in - cannot for non host objects
>>> delete - cannot for non host objects
>>> enumerate - cannot for non host objects
>>> hasOwnProperty - cannot for non host objects
>>>
>>> Incidentally, is Object.prototype.hasOwnProperty(myProxy)
>>> O(myProxyHandler.keys().length) for proxies?  This seems bad since a
>>> for (in) loop that filters out non-own properties would be O(n**2)  
>>> on
>>> top of the loop body.
>>>
>>> If a future version of ES includes some kind of generator/iterator
>>> scheme, then "in" and "enumerate" cease to be proxy specific.  But
>>> iterators could be implemented as proxies if instead of providing
>>> (has, keys, enumerate) we provide (prop, keyProducer) where
>>> prop(property) -> one of (undefined, OWN, INHERITED), and  
>>> keyProducer
>>> returns a function, that each time it's called returns a string
>>> property name or undefined to signal no-more.  Of course, trying to
>>> freeze an object that returns a key name multiple times is hard to
>>> define, but returning an array has the same problem.
>>>
>>> If there is a lazy key mechanism then iterators can be implemented  
>>> as proxies
>>>    function iterator(producer) {
>>>      var pending, produced = false;
>>>      function fetch() {
>>>        if (produced) { return; }
>>>        try {
>>>          pending = producer(); produced = true;
>>>        } catch (e) {
>>>          if (e !== NO_MORE_ELEMENTS) { throw e; }
>>>        }
>>>      }
>>>      return Proxy.create({
>>>          get: function (property) {
>>>            if ('next' === property) {
>>>              fetch();
>>>              var result = pending;
>>>              pending, produced = void 0, false;
>>>            }
>>>          },
>>>         has: function (property) {
>>>           return 'next' === property && (fetch(), produced);
>>>         },
>>>         keyProducer: function () {
>>>           return function () { return fetch(), produced ? 'next' :  
>>> void 0;
>>>         }
>>>       });
>>>    }
>>> This doesn't solve generators, since the pausing semantics of yield
>>> can't be easily implemented on top of proxies.
>>
>> Actually, a getter that can delete its own next property is all you
>> need for iterators.
>>
>> function iterator(producer) {
>>  var pending, produced;
>>  fetch();
>>  if (!produced) { return {}; }
>>  var it = {
>>    get next() {
>>      var result = pending;
>>      try {
>>        pending = producer();
>>        produced = true;
>>      } catch (e) {
>>        if (e !== STOP_ITERATION) { throw e; }
>>        delete it.next;
>>      }
>>      return result;
>>    }
>>  };
>>  return it;
>> }
>>
>> for (var it = iterator(x), item; 'next' in it;) {
>>  item = it.next;
>>  ...
>> }
>>
>>
>>>
>>> 2009/12/9 Mark S. Miller <erights at google.com>:
>>>> On Wed, Dec 9, 2009 at 11:02 AM, Brendan Eich  
>>>> <brendan at mozilla.com> wrote:
>>>>>
>>>>> On Dec 7, 2009, at 4:11 PM, Tom Van Cutsem wrote:
>>>>>
>>>>> Dear all,
>>>>>
>>>>> Over the past few weeks, MarkM and myself have been working on a  
>>>>> proposal
>>>>> for catch-alls for ES-Harmony based on proxies. I just uploaded  
>>>>> a strawman
>>>>> proposal to the wiki:
>>>>>
>>>>> http://wiki.ecmascript.org/doku.php?id=strawman:proxies
>>>>>
>>>>> Hi Tom, great to see this proposal. I took the liberty of making  
>>>>> a few
>>>>> small edits; hope they're ok. I like the stratification and the  
>>>>> ab-initio
>>>>> nature of the design -- the last seems to me to be a crucial  
>>>>> improvement
>>>>> over past proposals, which may help overcome the "climbing the  
>>>>> meta ladder"
>>>>> objection.
>>>>> Some initial comments, pruned to avoid restating others' comments:
>>>>> 1. This proposal obligates the catch-all implementor to delegate  
>>>>> to any
>>>>> prototype object in has and get, to include unshadowed prototype  
>>>>> properties
>>>>> in enumerate, to shadow if p in receiver.[[Prototype]] in put,  
>>>>> and to do
>>>>> nothing for delete proxy.p if !proxy.hasOwnProperty(p).
>>>>> In general, handler writers have to implement standard prototype- 
>>>>> based
>>>>> delegation if it is desired. This is probably the right thing,  
>>>>> but I wonder
>>>>> if you considered the alternative where prototype delegation is  
>>>>> handled "by
>>>>> the spec" or "by the runtime" and the proxy is considered "flat"?
>>>>
>>>> We did think about it, but it seemed needlessly less flexible. If  
>>>> such
>>>> flat-and-delegate handling is desired, an abstraction can be  
>>>> built on top of
>>>> ours that emulates it as a convenience. The reverse emulation seems
>>>> difficult at best.
>>>>
>>>>
>>>>>
>>>>> 2. The fix handler returning undefined instead of throwing  
>>>>> explicitly to
>>>>> reject a freeze, etc., attempt is a bit implicit. Falling off  
>>>>> the end of the
>>>>> function due to a forgetten or bungled return will do this. Ok,  
>>>>> let's say
>>>>> the programmer will test and fix the bug.
>>>>> But more significant: could there be a useful default denoted by  
>>>>> returning
>>>>> undefined or falling off the end of the fix function? An  
>>>>> alternative
>>>>> interpretation would be an empty frozen object. This has  
>>>>> symmetry with
>>>>> undefined passed (or no actual argument supplied) to  
>>>>> Object.create. It's a
>>>>> minor comment for sure.
>>>>
>>>> Since the undefined may be the result of a bug as you say, it  
>>>> seems worse
>>>> for the bug to silently result in fixing the proxy into an empty  
>>>> frozen
>>>> object. We think the current "noisy" behavior better supports  
>>>> defensive
>>>> programming.
>>>>
>>>>>
>>>>> 3. Mozilla's wrappers (proxies, membranes), which we pioneered for
>>>>> security purposes (e.g. for DOM inspectors where privileged JS is
>>>>> interacting with web content objects) and which have been copied  
>>>>> in other
>>>>> browsers (at least WebKit), implement === by unwrapping, so two  
>>>>> wrappers for
>>>>> the same object are === with that object, and with each other.
>>>>
>>>> In answer,
>>>> <http://wiki.ecmascript.org/doku.php?id=strawman:proxies#an_identity-preserving_membrane 
>>>> >
>>>> preserves === correspondence on each side of a membrane. Now that  
>>>> we have a
>>>> concrete catchall proposal adequate to build membranes, we'd like  
>>>> to restart
>>>> our discussions with Mozilla (JetPack, etc) about whether you  
>>>> could rebuild
>>>> some of your C++ membranes in JS code using these primitives. We  
>>>> should
>>>> follow up offlist.
>>>>
>>>>>
>>>>> The proxies proposal does not have an unwrapped object, although  
>>>>> super? is
>>>>> similar. Later in the proposal, you write "meta-level code will  
>>>>> ‘see’ the
>>>>> proxy rather than the object it represents." This sounds more  
>>>>> like wrappers
>>>>> as we use them -- there is always a wrapped object and its proxy  
>>>>> or wrapper.
>>>>> The alternative of not trapping === is a leaky abstraction that  
>>>>> inevitably
>>>>> breaks some programmers' expectations. Our early wrappers did  
>>>>> not hook ===,
>>>>> but eventually we settled on the unwrap-before-=== behavior  
>>>>> based on
>>>>> testing.
>>>>>
>>>>> This is a use-case I wanted to bring to your attention (Mike  
>>>>> Samuel raised
>>>>> it in his reply by suggesting a Proxy.proxies predicate; his  
>>>>> [[Class]]
>>>>> question also gets to the broader issue of transparency vs.  
>>>>> leaky proxy
>>>>> abstractions). Our wrapper experience suggests allowing === to  
>>>>> be hooked in
>>>>> a constrained way, for certain kinds of proxies. It could be  
>>>>> that this
>>>>> use-case can't be served by a standardized, general proxy/catch- 
>>>>> all
>>>>> proposal, and must be done under the hood and outside of the ES  
>>>>> spec.
>>>>
>>>> To avoid some "climbing meta ladder" issues, we purposely  
>>>> distinguish
>>>> between what we consider base-level operations, such as x.foo, and
>>>> meta-level operations, such as Object.getOwnProperty(x, 'foo').  
>>>> We attempt
>>>> to be as fully transparent (leak free) as reasonably possible at
>>>> virtualizing base level operations. We attempt to be fully non- 
>>>> transparent
>>>> (leak like a firehose) to meta-level operations. Some of our  
>>>> classification
>>>> may seem weird: Object.prototype.toString() is meta-level. It can  
>>>> be used to
>>>> reveal that an object is a trapping proxy.  
>>>> Object.getOwnPropertyNames() is
>>>> meta-level. Object.keys() is base level.
>>>> The properties of === that we feel need to be preserved:
>>>> 1) "x === y" does not cause any user code to run.
>>>> 2) "x === y" neither gives x access to y nor vice versa.
>>>> 3) "typeof x !== 'number' && x === y" mean that x is  
>>>> operationally identical
>>>> to y in all observable ways. Substituting the value for x with  
>>>> the value of
>>>> y cannot change the meaning of a computation.
>>>> 4) "x === y" implies that the Grant Matcher
>>>> <http://erights.org/elib/equality/grant-matcher/> may safely send  
>>>> the money
>>>> to either x or y.
>>>> A wrapper is not identical to the object it is wrapping, or there  
>>>> wouldn't
>>>> be any point in wrapping it. Thus, they can't be ===.
>>>> Two independent wrappers on the same object may behave differently,
>>>> depending on their definers. Thus they can't be ===.
>>>> Except for two proxies with identical parts, such as two object  
>>>> proxies with
>>>> identical handlers and supers. However, as shown by our example  
>>>> membrane,
>>>> one can just use an Ephemeron table to avoid creating  
>>>> semantically identical
>>>> duplicate proxies, preserving === without magic.
>>>>
>>>>>
>>>>> 4. The [[Get]] versus [[Invoke]] rationale: indeed performance  
>>>>> is a
>>>>> concern, but existing engines also specialize callee-computation  
>>>>> distinctly
>>>>> from get-value, in order to optimize away Reference types. The  
>>>>> ES specs so
>>>>> far do not, instead using the internal Reference type to delay  
>>>>> GetValue so
>>>>> as to bind |this| to the Reference base when computing a callee  
>>>>> and its
>>>>> receiver as part of evaluating a call expression.
>>>>>  I think it is an open question whether a future spec,  
>>>>> especially one
>>>>> using a definitional interpreter, will stick to References. If  
>>>>> we end up
>>>>> making the distinction that all practical implementations  
>>>>> already make,
>>>>> between get-as-part-of-callee-computation and all other get- 
>>>>> value "gets",
>>>>> then I don't think this rationale is so strong.
>>>>> In general over-coupling to ES5 may not help either a new  
>>>>> Harmony-era
>>>>> proposal to "get in", or to be as complete or expressive as it  
>>>>> should be. So
>>>>> a rationale based on choices or limitations of ES1-5 seems weak  
>>>>> to me.
>>>>
>>>>
>>>> +100.
>>>> I would love to see the concept of References disappear, and to  
>>>> see the
>>>> ([[Get]], [[Call]]) pairs in the spec that really mean "call this  
>>>> as a
>>>> method" be rewritten as [[Invoke]]s. In that case, I would  
>>>> enthusiastically
>>>> agree that this catchall proposal should be upgraded with an  
>>>> invoke() trap.
>>>> Note how this would make our membrane code simpler and more  
>>>> efficient.
>>>> Rather than a get() trap at the choke point that creates and  
>>>> returns a
>>>> function, we'd simply have an invoke() trap whose body is that  
>>>> function's
>>>> body.
>>>> I'd like to understand better how we could get rid of References.
>>>>
>>>>>
>>>>> Thanks again for this proposal,
>>>>
>>>> You're welcome. It was fun!
>>>>
>>>> --
>>>>    Cheers,
>>>>    --MarkM
>>>>
>>>> _______________________________________________
>>>> es-discuss mailing list
>>>> es-discuss at mozilla.org
>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>
>>>>
>>>
>>



More information about the es-discuss mailing list