Catch-all proposal based on proxies

Mike Samuel mikesamuel at gmail.com
Thu Dec 10 11:10:18 PST 2009


2009/12/10 Brendan Eich <brendan at mozilla.com>:
> On Dec 10, 2009, at 9:57 AM, Mike Samuel wrote:
>
>> Actually, I think if iterators are desired, proxies may be a good way.
>
> We have implemented iterators and generators since 2006 in JS1.7 (Firefox
> 2), and they do not involve proxies or property mutation including deletion.
> Those are costly features and I don't see why they are necessary for an
> iteration protocol.
>
> var examp1 = {a:1, b:2, c:3};
> var examp2 = [4, 5, 6];
>
> function keyIterator() {
>    let keys = Object.keys(this);

This will fail for infinite iterators.

>    let i = 0;
>    return {
>        next: function () {
>            if (i == keys.length)
>                throw StopIteration;
>            return keys[i++];
>        }
>    };
> }
>
> function indexIterator() {
>    let self = this;
>    let i = 0;
>    return {
>        next: function () {
>            if (i == self.length)
>                throw StopIteration;
>            return i++;
>        }
>    };
> }
>
> Object.defineIterator(examp1, keyIterator);
> Object.defineIterator(examp2, indexIterator);
>
> for (let i in examp1)
>    print(i + " (type " + typeof i + ")");
>
> for (let i in examp2)
>    print(i + " (type " + typeof i + ")");
>
> ---
>
> example output:
>
> a (type string)
> b (type string)
> c (type string)
> 0 (type number)
> 1 (type number)
> 2 (type number)
>
> ---
>
> Notes:
>
> 1. After Python, the meta-level handler is a function that returns an
> iterator for its receiver object. Yeah, that means |this| -- but it's just a
> convention. The handler could take an explicit obj parameter instead.
>
> 2. After Python, an iterator is an object with a next() method returning the
> next value in the iteration, or throwing StopIteration to end the iteration.
>
> 3. The StopIteration object is a well-known singleton (like Math). In our
> implementation each frame has one but any will do to terminate iteration
> (they all have the same [[Class]]).
>
> /be
>
> P.S. Here are some shims from JS1.[78] in SpiderMonkey to ES5 plus
> defineIterator:
>
> Object.defineProperty = function (obj, name, desc) {
>    return obj.__defineProperty__(name, desc.value,
>                                  !desc.configurable, !desc.writable,
> !desc.enumerable);
> };
>
> Object.defineIterator = function (obj, iter) {
>    Object.defineProperty(obj, '__iterator__', {value: iter});
> };
>
>> Code below shows a problem where having an object delete it's own
>> property means hard choices have to be made over when to throw an
>> exception.  Proxies are fundamentally lazier and so dodge this issue:
>>
>> var myIterator = iterator(function () {
>>  var i = 0;
>>  return function () {
>>   if (i === 10) throw STOP_ITERATION;
>>   return i++; };
>>  }());
>>
>> while ('next' in myIterator) { alert(myIterator.next); }
>>
>> var STOP_ITERATION = {};
>>
>> function iterator(producer) {
>>  var produced, pending;
>>  function fetch() {
>>   if (produced) { return; }
>>   try {
>>     pending = producer();
>>   } catch (e) {
>>     if (e !== STOP_ITERATION) { throw e; }
>>     return;
>>   }
>>   produced = true;
>>  }
>>  fetch();
>>  if (!produced) { return {}; }
>>  var it = {
>>   get next() {
>>     var result = pending;
>>     pending = void 0;
>>     produced = false;
>>     try {
>>       fetch(); // throws too early?
>>     } finally {
>>       if (!produced) { delete it.next; }  // throws if frozen losing
>> result
>>     }
>>     return result;
>>   }
>>  };
>>  return it;
>> }
>>
>>
>>
>> 2009/12/10 Mike Samuel <mikesamuel at gmail.com>:
>>>
>>> 2009/12/9 Mike Samuel <mikesamuel at gmail.com>:
>>>>
>>>> On the climbing the meta, I'd like to understand how this might
>>>> interact with other proposals.
>>>>
>>>> get - already can execute arbitrary code due to getters
>>>> set - already can execute arbitrary code due to setters
>>>> in - cannot for non host objects
>>>> delete - cannot for non host objects
>>>> enumerate - cannot for non host objects
>>>> hasOwnProperty - cannot for non host objects
>>>>
>>>> Incidentally, is Object.prototype.hasOwnProperty(myProxy)
>>>> O(myProxyHandler.keys().length) for proxies?  This seems bad since a
>>>> for (in) loop that filters out non-own properties would be O(n**2) on
>>>> top of the loop body.
>>>>
>>>> If a future version of ES includes some kind of generator/iterator
>>>> scheme, then "in" and "enumerate" cease to be proxy specific.  But
>>>> iterators could be implemented as proxies if instead of providing
>>>> (has, keys, enumerate) we provide (prop, keyProducer) where
>>>> prop(property) -> one of (undefined, OWN, INHERITED), and keyProducer
>>>> returns a function, that each time it's called returns a string
>>>> property name or undefined to signal no-more.  Of course, trying to
>>>> freeze an object that returns a key name multiple times is hard to
>>>> define, but returning an array has the same problem.
>>>>
>>>> If there is a lazy key mechanism then iterators can be implemented as
>>>> proxies
>>>>   function iterator(producer) {
>>>>     var pending, produced = false;
>>>>     function fetch() {
>>>>       if (produced) { return; }
>>>>       try {
>>>>         pending = producer(); produced = true;
>>>>       } catch (e) {
>>>>         if (e !== NO_MORE_ELEMENTS) { throw e; }
>>>>       }
>>>>     }
>>>>     return Proxy.create({
>>>>         get: function (property) {
>>>>           if ('next' === property) {
>>>>             fetch();
>>>>             var result = pending;
>>>>             pending, produced = void 0, false;
>>>>           }
>>>>         },
>>>>        has: function (property) {
>>>>          return 'next' === property && (fetch(), produced);
>>>>        },
>>>>        keyProducer: function () {
>>>>          return function () { return fetch(), produced ? 'next' : void
>>>> 0;
>>>>        }
>>>>      });
>>>>   }
>>>> This doesn't solve generators, since the pausing semantics of yield
>>>> can't be easily implemented on top of proxies.
>>>
>>> Actually, a getter that can delete its own next property is all you
>>> need for iterators.
>>>
>>> function iterator(producer) {
>>>  var pending, produced;
>>>  fetch();
>>>  if (!produced) { return {}; }
>>>  var it = {
>>>   get next() {
>>>     var result = pending;
>>>     try {
>>>       pending = producer();
>>>       produced = true;
>>>     } catch (e) {
>>>       if (e !== STOP_ITERATION) { throw e; }
>>>       delete it.next;
>>>     }
>>>     return result;
>>>   }
>>>  };
>>>  return it;
>>> }
>>>
>>> for (var it = iterator(x), item; 'next' in it;) {
>>>  item = it.next;
>>>  ...
>>> }
>>>
>>>
>>>>
>>>> 2009/12/9 Mark S. Miller <erights at google.com>:
>>>>>
>>>>> On Wed, Dec 9, 2009 at 11:02 AM, Brendan Eich <brendan at mozilla.com>
>>>>> wrote:
>>>>>>
>>>>>> On Dec 7, 2009, at 4:11 PM, Tom Van Cutsem wrote:
>>>>>>
>>>>>> Dear all,
>>>>>>
>>>>>> Over the past few weeks, MarkM and myself have been working on a
>>>>>> proposal
>>>>>> for catch-alls for ES-Harmony based on proxies. I just uploaded a
>>>>>> strawman
>>>>>> proposal to the wiki:
>>>>>>
>>>>>> http://wiki.ecmascript.org/doku.php?id=strawman:proxies
>>>>>>
>>>>>> Hi Tom, great to see this proposal. I took the liberty of making a few
>>>>>> small edits; hope they're ok. I like the stratification and the
>>>>>> ab-initio
>>>>>> nature of the design -- the last seems to me to be a crucial
>>>>>> improvement
>>>>>> over past proposals, which may help overcome the "climbing the meta
>>>>>> ladder"
>>>>>> objection.
>>>>>> Some initial comments, pruned to avoid restating others' comments:
>>>>>> 1. This proposal obligates the catch-all implementor to delegate to
>>>>>> any
>>>>>> prototype object in has and get, to include unshadowed prototype
>>>>>> properties
>>>>>> in enumerate, to shadow if p in receiver.[[Prototype]] in put, and to
>>>>>> do
>>>>>> nothing for delete proxy.p if !proxy.hasOwnProperty(p).
>>>>>> In general, handler writers have to implement standard prototype-based
>>>>>> delegation if it is desired. This is probably the right thing, but I
>>>>>> wonder
>>>>>> if you considered the alternative where prototype delegation is
>>>>>> handled "by
>>>>>> the spec" or "by the runtime" and the proxy is considered "flat"?
>>>>>
>>>>> We did think about it, but it seemed needlessly less flexible. If such
>>>>> flat-and-delegate handling is desired, an abstraction can be built on
>>>>> top of
>>>>> ours that emulates it as a convenience. The reverse emulation seems
>>>>> difficult at best.
>>>>>
>>>>>
>>>>>>
>>>>>> 2. The fix handler returning undefined instead of throwing explicitly
>>>>>> to
>>>>>> reject a freeze, etc., attempt is a bit implicit. Falling off the end
>>>>>> of the
>>>>>> function due to a forgetten or bungled return will do this. Ok, let's
>>>>>> say
>>>>>> the programmer will test and fix the bug.
>>>>>> But more significant: could there be a useful default denoted by
>>>>>> returning
>>>>>> undefined or falling off the end of the fix function? An alternative
>>>>>> interpretation would be an empty frozen object. This has symmetry with
>>>>>> undefined passed (or no actual argument supplied) to Object.create.
>>>>>> It's a
>>>>>> minor comment for sure.
>>>>>
>>>>> Since the undefined may be the result of a bug as you say, it seems
>>>>> worse
>>>>> for the bug to silently result in fixing the proxy into an empty frozen
>>>>> object. We think the current "noisy" behavior better supports defensive
>>>>> programming.
>>>>>
>>>>>>
>>>>>> 3. Mozilla's wrappers (proxies, membranes), which we pioneered for
>>>>>> security purposes (e.g. for DOM inspectors where privileged JS is
>>>>>> interacting with web content objects) and which have been copied in
>>>>>> other
>>>>>> browsers (at least WebKit), implement === by unwrapping, so two
>>>>>> wrappers for
>>>>>> the same object are === with that object, and with each other.
>>>>>
>>>>> In answer,
>>>>>
>>>>> <http://wiki.ecmascript.org/doku.php?id=strawman:proxies#an_identity-preserving_membrane>
>>>>> preserves === correspondence on each side of a membrane. Now that we
>>>>> have a
>>>>> concrete catchall proposal adequate to build membranes, we'd like to
>>>>> restart
>>>>> our discussions with Mozilla (JetPack, etc) about whether you could
>>>>> rebuild
>>>>> some of your C++ membranes in JS code using these primitives. We should
>>>>> follow up offlist.
>>>>>
>>>>>>
>>>>>> The proxies proposal does not have an unwrapped object, although
>>>>>> super? is
>>>>>> similar. Later in the proposal, you write "meta-level code will ‘see’
>>>>>> the
>>>>>> proxy rather than the object it represents." This sounds more like
>>>>>> wrappers
>>>>>> as we use them -- there is always a wrapped object and its proxy or
>>>>>> wrapper.
>>>>>> The alternative of not trapping === is a leaky abstraction that
>>>>>> inevitably
>>>>>> breaks some programmers' expectations. Our early wrappers did not hook
>>>>>> ===,
>>>>>> but eventually we settled on the unwrap-before-=== behavior based on
>>>>>> testing.
>>>>>>
>>>>>> This is a use-case I wanted to bring to your attention (Mike Samuel
>>>>>> raised
>>>>>> it in his reply by suggesting a Proxy.proxies predicate; his [[Class]]
>>>>>> question also gets to the broader issue of transparency vs. leaky
>>>>>> proxy
>>>>>> abstractions). Our wrapper experience suggests allowing === to be
>>>>>> hooked in
>>>>>> a constrained way, for certain kinds of proxies. It could be that this
>>>>>> use-case can't be served by a standardized, general proxy/catch-all
>>>>>> proposal, and must be done under the hood and outside of the ES spec.
>>>>>
>>>>> To avoid some "climbing meta ladder" issues, we purposely distinguish
>>>>> between what we consider base-level operations, such as x.foo, and
>>>>> meta-level operations, such as Object.getOwnProperty(x, 'foo'). We
>>>>> attempt
>>>>> to be as fully transparent (leak free) as reasonably possible at
>>>>> virtualizing base level operations. We attempt to be fully
>>>>> non-transparent
>>>>> (leak like a firehose) to meta-level operations. Some of our
>>>>> classification
>>>>> may seem weird: Object.prototype.toString() is meta-level. It can be
>>>>> used to
>>>>> reveal that an object is a trapping proxy. Object.getOwnPropertyNames()
>>>>> is
>>>>> meta-level. Object.keys() is base level.
>>>>> The properties of === that we feel need to be preserved:
>>>>> 1) "x === y" does not cause any user code to run.
>>>>> 2) "x === y" neither gives x access to y nor vice versa.
>>>>> 3) "typeof x !== 'number' && x === y" mean that x is operationally
>>>>> identical
>>>>> to y in all observable ways. Substituting the value for x with the
>>>>> value of
>>>>> y cannot change the meaning of a computation.
>>>>> 4) "x === y" implies that the Grant Matcher
>>>>> <http://erights.org/elib/equality/grant-matcher/> may safely send the
>>>>> money
>>>>> to either x or y.
>>>>> A wrapper is not identical to the object it is wrapping, or there
>>>>> wouldn't
>>>>> be any point in wrapping it. Thus, they can't be ===.
>>>>> Two independent wrappers on the same object may behave differently,
>>>>> depending on their definers. Thus they can't be ===.
>>>>> Except for two proxies with identical parts, such as two object proxies
>>>>> with
>>>>> identical handlers and supers. However, as shown by our example
>>>>> membrane,
>>>>> one can just use an Ephemeron table to avoid creating semantically
>>>>> identical
>>>>> duplicate proxies, preserving === without magic.
>>>>>
>>>>>>
>>>>>> 4. The [[Get]] versus [[Invoke]] rationale: indeed performance is a
>>>>>> concern, but existing engines also specialize callee-computation
>>>>>> distinctly
>>>>>> from get-value, in order to optimize away Reference types. The ES
>>>>>> specs so
>>>>>> far do not, instead using the internal Reference type to delay
>>>>>> GetValue so
>>>>>> as to bind |this| to the Reference base when computing a callee and
>>>>>> its
>>>>>> receiver as part of evaluating a call expression.
>>>>>>  I think it is an open question whether a future spec, especially one
>>>>>> using a definitional interpreter, will stick to References. If we end
>>>>>> up
>>>>>> making the distinction that all practical implementations already
>>>>>> make,
>>>>>> between get-as-part-of-callee-computation and all other get-value
>>>>>> "gets",
>>>>>> then I don't think this rationale is so strong.
>>>>>> In general over-coupling to ES5 may not help either a new Harmony-era
>>>>>> proposal to "get in", or to be as complete or expressive as it should
>>>>>> be. So
>>>>>> a rationale based on choices or limitations of ES1-5 seems weak to me.
>>>>>
>>>>>
>>>>> +100.
>>>>> I would love to see the concept of References disappear, and to see the
>>>>> ([[Get]], [[Call]]) pairs in the spec that really mean "call this as a
>>>>> method" be rewritten as [[Invoke]]s. In that case, I would
>>>>> enthusiastically
>>>>> agree that this catchall proposal should be upgraded with an invoke()
>>>>> trap.
>>>>> Note how this would make our membrane code simpler and more efficient.
>>>>> Rather than a get() trap at the choke point that creates and returns a
>>>>> function, we'd simply have an invoke() trap whose body is that
>>>>> function's
>>>>> body.
>>>>> I'd like to understand better how we could get rid of References.
>>>>>
>>>>>>
>>>>>> Thanks again for this proposal,
>>>>>
>>>>> You're welcome. It was fun!
>>>>>
>>>>> --
>>>>>   Cheers,
>>>>>   --MarkM
>>>>>
>>>>> _______________________________________________
>>>>> es-discuss mailing list
>>>>> es-discuss at mozilla.org
>>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>>
>>>>>
>>>>
>>>
>
>


More information about the es-discuss mailing list