yield syntax (diverging from: How would shallow generators compose with lambda?)

[Brendan Eich]

This may be breaking a butterfly on a wheel, but I am game if it  
improves the state of the strawman proposals.

On May 15, 2009, at 7:16 PM, David-Sarah Hopwood wrote:

> My point was that the example of 'yield (foo)' (that is, yield as a
> prefix operator applied to the expression '(foo)') shows that the  
> prefix
> operator syntax cannot possibly be easier to specify than the function
> call syntax -- contrary to what you appeared to be arguing above.

Analogous to direct vs. indirect eval in ES5 (15.1.2.1.1), there is no  
purely syntactic specification for what Neil proposes. A runtime check  
is required. So I don't see why you are focusing only on syntax here.

All the costs count, but considering "who pays" can trump "how much";  
see the postscript below. I'm more concerned with usability than  
implementability, but the latter is not hard and there's no soundness  
issue.


> In fact I think it is much harder to do correctly.

See below, it's quite easy.


>> Yes, but it's not that complicated. SpiderMonkey and Rhino do it.  
>> Code
>> size burden is in the noise.
>
> Hmm. SpiderMonkey and Rhino use ad-hoc parsers. Show me an unambiguous
> grammar for the prefix yield operator, and then I'll concede the  
> point :-)

Python 2.5 has essentially the same issues, and it has a formal  
grammar (several, the one CPython uses and at least one other at  
antlr.org). From CPython's Grammar/Grammar EBNF:

$ grep yield Grammar
expr_stmt: testlist (augassign (yield_expr|testlist) |
                      ('=' (yield_expr|testlist))*)
flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt |  
yield_stmt
yield_stmt: yield_expr
atom: ('(' [yield_expr|testlist_gexp] ')' |
yield_expr: 'yield' [testlist]

What we are discussing is almost trivial in terms of usability, but  
the difference is big enough to care about, IMHO. Either you have to  
write, in Neil's approach:

yield (E)      (or just yield() to yield undefined)

or in the Pythonic approach:

(yield E)      (or just (yield))

with the parentheses in the Pythonic approach not required when the  
yield expression is either (a) the entire expression in an expression  
statement; (b) the right-hand side of an assignment.

Python differs from JS by having only assignment statements, not  
assignment expressions, which simplifies the problem a bit, as the  
grep output cited above shows.

In any case, yield E; as a statement is the most common kind of yield  
in real-world Python code, both because it came first and because  
sending values to generators is less commonly done than iterating  
generators.

So with either approach you have some "insignificant, silly  
parentheses" -- but with Neil's approach you have "lots" more because  
the expression-statement yield must have at least () after the keyord.

This tips the usability contest in favor of the Pythonic approach.

In no case are there ambiguities for a bottom up grammar. The right  
parenthesis avoids nasty automatic semicolon insertion problems that  
plagued the unbracketed low-precedence expression on the right of a  
high-precedence prefix in the case of expression-bodied functions, AKA  
"expression closures" (see https://mail.mozilla.org/pipermail/es-discuss/2008-October/007888.html) 
.

To confirm this, I patched WebKit's Bison grammar for JS to support  
yield //a la// JS1.7. I didn't hook up the semantic actions, just  
verified zero reduce-reduce conflicts. Patch attached below --  
comments welcome (including from you WebKit lurkers :-).

/be

P.S. Bean-counting productions in a bottom-up grammar is counting the  
wrong cost, given the few implementors and their greater skills,  
compared to the many users who need to be spared those painful and  
fruitless parentheses.

Bean-counting productions in a traditional bottom-up grammar is also  
silly given how many are required for ES1-3. From the patch, notice  
all the existing specialized NoIn/NoBF/NoNode variants in parser/ 
Grammar.y (NoBF = No Brace at Front, as far as I can tell).

Much of that production-forking could be avoided by using either  
parameterized LR parsing (www.cs.lth.se/Research/LDTA2004/d09_TheimannNeubauer.pdf) 
, or else top-down ("ad-hoc" or not :-P) parsing.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch
Type: application/octet-stream
Size: 8785 bytes
Desc: not available
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20090517/eb8f4608/attachment-0001.obj>
-------------- next part --------------