lexical for-in/for-of loose end

Allen Wirfs-Brock allen at wirfs-brock.com
Thu Feb 2 11:35:38 PST 2012


On Feb 1, 2012, at 7:13 PM, Brendan Eich wrote:

> Allen Wirfs-Brock wrote:
>> On Feb 1, 2012, at 3:31 PM, Brendan Eich wrote:
>>> Brendan Eich wrote:
> ...
>> That raises the question of the role of the for-in statement in the language (besides just being an obsolete and less useful alternative to for-of).  Being a full fledged syntactic statement of the language, it certainly isn't hiding in an obscure corner.  If we want to think of it as something other than an over exposed reflection tool, what is it for?.  The best expression of this that I can come up with, is that for-in is a statement that is primarily useful for iterating the "data member keys" of simple record and array like data abstractions.

I probably should have included "particularly those created using JSON.parse" at the end of the last sentence above.
> 
> So far, so good, and latent here is the best case for not elaborating the left-hand side to include destructuring, if you can avoid bloating the grammar too much. Contra my "regularity plus small use-cases" argument, we leave for-in alone. We can even leave it underspecified as in ES5.

but still loose the initializer on var, right?

Also, for consistency I still allow let/const in place or var with with fresh bindings on each iterations.  These are really orthogonal issues relating to the new blocked scoped declaration and we should handle them constantly everywhere. 

> 
>> From that perspective,  own-ness again is something that should primarily be an implementation concern.
> 
> Historically this came up because pre-ES5 people couldn't extend Array.prototype, e.g., without making enumerable properties. But of course then the wisdom said don't use for-in on arrays. Object.prototype is verboten, and at some cost in using custom functional-programming style iteration, we're ok (PrototypeJS status quo).
> 
> The problem that may remain is that
> 
>  for own (k in o) ...;
> 
> still beats
> 
>  for (k in keys(o)) ...;
> 
> not only by a couple of chars not counting import or assuming prelude, but in terms of people's historical memory and folk-wisdom learning.

I would think we could make this 
   for (k in own(o)) ...;
and
   for (k of own(o)) ...;
if the object produced by own() provides appropiate definitions for [[Enumerate]] and [[Iterate]]

Presumably, the only prior leaning that applies comes from CoffeeScript.  While CS is popular, it isn't at all close to being used by the majority of JS developers.  CS experience is informative but I don't think we have to worry so much about CS derived habits.

> 
> Should we ignore all this and say "just use for-of with the right iterator?"

probably...

> 
>> To me, for-own-in makes it too easy for client code to create dependencies upon what should inconsequential implementation decisions.
> 
> In reality the shoe is on the other foot. Implementation decisions that should be without consequence mess up for-in, requiring hasOwnProperty testing. People want a shorter-path solution.
> 
> Giving a longer-path solution with for-of and iterators is good, we want that. Is it good enough to be the only solution we build into the language?

If we agree that for-in is "the problem", then it seems like we should be trying to make for-of more attractive while at the same time making for-in less attractive (or at least not improving for-in in a way that also makes it more attractive)

> 
>> We have a goal of making ES a language that is better for creating and using abstractions.  For-own-in is a tool that tunnels through abstractions.
> 
> I think you're making too sweeping a statement about abstractions. Does for-own-in tunnel in ways that violate abstractions, or is it really what the doctor ordered for JS close to today's best practices (not always followed)? I say more the latter, but not clear cut in any event.

JS today isn't a very good language for building abstractions of the sort that are useful for structuring complex applications and correcting that is one of our goals for Harmony.  Today's JS best practices don't really reflect this perspective and we should be careful about enshrining backwards looking practices.

> 
>> Such a tool is fine as part of the reflection facility, but making it a full fledged statement seems like it would be creating an attractive nuisance.
> 
> We have a full-fledged statement, for-in. It can't be deprecated quite yet. Adding for-of helps, but is it enough? That's the issue I'm raising.
> 
> ...
> 
>>> Our current position is "use for-of and an items helper function":
>>> 
>>> import items from "@reflect"; // currently @iter; could be part of standard prelude
>>> 
>>> for ([k, v] of items(o)) ...;
>> 
>> see the main part of the message I referenced above. Pure functional filters like item are unfriendly to collection abstracton builders.
> 
> JS has little of the OO collection building you see in Smalltalk or Java. Is this a bug in the language and communities using it? I'm not so sure.

I think this is primarily a reflection of JS poor support for abstraction, in general. If it hard to build a basic entity abstraction very few people are going to get around to creating collection abstractions.

In fact, at the root of what we are discussing here (enumeration vs iteration, own-ess, etc) are issues relating to how you generalize over various sorts of collection abstractions.  Hardwired iteration statements make it difficult to do this.  They end of with lots of knobs and switches to handle various special cares and then still don't sufficiently generalize.  On the other hand,  there are well established OO techniques for this.  Their primary advantage is that they place the complication in the hands to the collection implementor rather than the collection client. 

> 
> Your point that we want functional helpers to delegate to OO methods is well-taken, and IIRC that's what Python does. The proposal at
> 
> http://wiki.ecmascript.org/doku.php?id=harmony:iterators#standard_api
> 
> does not delegate from items(d) to d. at items or whatever it ought to be called, but this is easy to add. Cc'ing Jason for his thoughts.
> 
> 
>>> But I think detailing the design of ES6 must be allowed to entertain an even easier-to-use extension to for-in.
>> 
>> Another simplification would be to make default iterator produce {key,value} item objects,
>> 
>> Then we have three concise formulations:
>> 
>> for ({key, value} of o) ...;
>> for  ({key} of o) ...;
>> for ({value} of o) ...
> 
> No, too expensive. We want just the keys or just the values, often enough. Narrow is better than wide.

probably, but I suspect that there may be optimization techniques (for example via a parameter to next) that could help the most common cases.

> 
> 
>>> If you want values not keys, then per the current proposal you use
>>> 
>>> for (v of values(o)) ...;
>>> 
>>> given the necessary values import, or inclusion in a standard prelude.
>>> 
>>> There's a hole vs. CoffeeScript: we do not want for-of on a plain Object to iterate over enumerable property values. Arrays, yes, Objects, no -- no Object.prototype. at iterator. So no |for (v of o) ...| for o = {p:1, q:2, r:3}.
>> (I note at this point that there are some huge es-discuss threads on topics such as the above for which the conclusions (if any) have not been extracted. I'm starting to go over them to see what I can extract from them)
>> 
>> I assume that you are arguing that the default @iterator for Object.prototype (however it is provided)
> 
> No! I clearly said there is no @iterator in Object.prototype.
> 
>> does a key enumeration just like ES1-5 for-in.  Or are you arguing that it produces nothing?
> 
> No @iterator in Object.prototype. We've been over this, Jason argued convincingly against pre-defining one because it is future-hostile to collection iteration.
I just need to find the right thread...

There is still a default.  The iterators proposal says that for-of falls back to @enumerate if @iterate isn't present. 

BTW, this is a good example of something that should not be built into statement semantics.  Why should |for (v or values(o))...| iterate differently than |values(o).forOf({|v|...})|

My currently thinking (working off of the current wiki proposal) is to have [[Enumerate]] and [[Iterate]] internal methods (for all objects) and it is in their default implementation that [[Iterate]] delegates to [[Enumerate]].

> 
> 
>> I'm not exactly sure why either of those is better than |for (v of o) ...| but I think there are a couple threads that I need to digest that drill into the issues.
> 
> No issue here, I think. We want for (x of {p:1, q:2}) to throw, without the user first having defined @iterator in Object.prototype (and that would not be recommended practice).

Not what the wiki proposal currently says...

> 
> 
>>> Also no for each (v in o) as E4X (ECMA-357) promulgated. SpiderMonkey and Rhino support it and probably will have to carry it, but such "each" does not say its meaning (values not keys) clearly, and it doesn't compose with "own" nicely.
>>> 
>>> But with destructuring for-in, you could write
>>> 
>>> for ([, v] in o) ...;
>> 
>> currently the grammar don't support the above hole syntax.  I wouldn't be a support for adding it.
> 
> Wait, holes are useful in destructuring array patterns, to avoid dummy bindings. We support holes in array destructuring in JS1.7 up. What is the problem? Saying the grammar doesn't support holes and you don't support them is no explanation.

Ok, I guess I missed that the grammar in the proposal page allowed such holes and none of the examples on the page show them.  this is a good example of why it is important for people to review the actual spec. drafts.

That said,  I'd hate to see things like:

   [,,,,,x] =  someArray;

and even
  
   [a,,y] = someArray;

can be easily missed.

It isn't clear to me that we are doing ES programmers (especially those with poor eyesight) a favor by allowing such expressions.  What is the evidence with JS1.7 up.  How much usage do we see of such holes?

> 
>>> This is a bit ugly (holes never look pretty, even when they're useful).
>>> 
>>> Not sure what I think of this but I thought I'd throw it out here on es-discuss. The reason I bring it up is twofold: 1) for own (k in o) still needs to be discussed; 2) the for ([k, v] of items(o)) ...; tax is a bit higher than I'd like.
>> 
>> and I'm trying to write spec. language for all of this right now, so it is a good time to make some decisions.
> 
> Quoting Jason's reply in the thread you cited:
> 
> To address Allen's original question:
> 
> I think the Map and Set classes in Harmony mean that not all
> structured data is stored as object properties. I think that's a good
> thing.
> 
> However it does mean that we must have a separation of concerns between
>  - iterating over a collection
>  - object property inspection

Almost sounds like the preamble for the Object Model Reformation proposal.  Yea, Jason...

> ...because I don't see how 'for (p of obj)' can be both the right
> syntax for walking an arbitrary object's properties *and* the right
> syntax for walking a Set's elements. Someday the "arbitrary object"
> will be a Set. And then what?

The way OMR approaches this problem (for [ ], but the basic idea applies here).  Is to consider basic objects to be a collection of properties that uses a default mapping of collection element access ([ ]) to property access (obj.prop). This preserves compatibility with ES1-5 while enabling user creation of collection with array-like element access.  I don't see why the same logic doesn't apply to for-of.

In particular, why we wouldn't want

    for (let [k,v] of items(JSON.parse(someString))) ...

to do the obvious thing.

If the default behavior for [[Iterate]] is to throw (rather than delegate to [[Enumerate]]), an implementer of Set (or any other collection abstraction) is still going to have to over-ride it.  The throw, in that case, fells a little nanny-ish.  Default to the classic interpretation of an ES object is a collection of properties seems preferable to me.

> 
> So if we take that separation of concerns as our maxim, what do we end
> up with? Here's a sketch.
> 
> Basics:
>  for (v of arr)  // each element of an Array
>  for (v of set)  // each value in a Set
>  for (k of map)  // each key in a Map, following Python

Smalltalk defaults maps to the value and Ruby to the key/value pair.  Java seems to have no default, you have to explicitly select keys, values or items.

The rationale for Smalltalk, is that most generic algorithms that want iterate over arbietrary collations are interested in values.  After all, some collections don't have keys.


> 
> Host objects / libraries:
>  for (elt of document.getElementsByTagName('P'))  // DOM
>  for (elt of $("div.main p"))  // hypothetical jQuery support
> 
> Other styles of iteration:
>  for ([i, v] of arr.items())  // index-value pairs (new Array method)
>  for ([k, v] of map.items())  // key-value pairs
>  for (v of map.values())  // just values, no keys (uncommon use case)
> 
> Enumerating properties:
>  import * from '@inspect';
>  for (p of keys(obj))
>  for (v of values(obj))
>  for ([p, v] of items(obj))
> Or if you don't want to import anything:
>  for (p of Object.keys(obj))
>  for (p of Object.getOwnPropertyNames(obj))
> 
> This makes Map slightly nicer to iterate over than Object. I think Map
> is a better, less error-prone Map than Object anyway, so that's
> appropriate.
> 
> -j
> 
> 
> I think we are close, but not quite where we should be. If we define keys, values, and items to work in terms of properties (they're currently spec'ed using for-in under the hood), are we missing a chance to make these universal for collections that have keys and values? Set can have values only, but the idea is to delegate from function to OO method (with private names @keys, @values, @items). Or do something better.
> 
> Less is more in our world, private name objects shedding syntax and getting into ES6 is one example. But we then talked about experience possibly justifying syntax too (I've used @ above). Could the same thing happen with iteration? I don't want to overdesign (who does? wait, stop...) but doing too little will just create obligatory library downloads.
> 
> /be
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20120202/1532063f/attachment-0001.html>


More information about the es-discuss mailing list