yield syntax (diverging from: How would shallow generators compose with lambda?)

Brendan Eich brendan at mozilla.com
Sun May 17 11:00:22 PDT 2009

This may be breaking a butterfly on a wheel, but I am game if it  
improves the state of the strawman proposals.

On May 15, 2009, at 7:16 PM, David-Sarah Hopwood wrote:

> My point was that the example of 'yield (foo)' (that is, yield as a
> prefix operator applied to the expression '(foo)') shows that the  
> prefix
> operator syntax cannot possibly be easier to specify than the function
> call syntax -- contrary to what you appeared to be arguing above.

Analogous to direct vs. indirect eval in ES5 (, there is no  
purely syntactic specification for what Neil proposes. A runtime check  
is required. So I don't see why you are focusing only on syntax here.

All the costs count, but considering "who pays" can trump "how much";  
see the postscript below. I'm more concerned with usability than  
implementability, but the latter is not hard and there's no soundness  

> In fact I think it is much harder to do correctly.

See below, it's quite easy.

>> Yes, but it's not that complicated. SpiderMonkey and Rhino do it.  
>> Code
>> size burden is in the noise.
> Hmm. SpiderMonkey and Rhino use ad-hoc parsers. Show me an unambiguous
> grammar for the prefix yield operator, and then I'll concede the  
> point :-)

Python 2.5 has essentially the same issues, and it has a formal  
grammar (several, the one CPython uses and at least one other at  
antlr.org). From CPython's Grammar/Grammar EBNF:

$ grep yield Grammar
expr_stmt: testlist (augassign (yield_expr|testlist) |
                      ('=' (yield_expr|testlist))*)
flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt |  
yield_stmt: yield_expr
atom: ('(' [yield_expr|testlist_gexp] ')' |
yield_expr: 'yield' [testlist]

What we are discussing is almost trivial in terms of usability, but  
the difference is big enough to care about, IMHO. Either you have to  
write, in Neil's approach:

yield (E)      (or just yield() to yield undefined)

or in the Pythonic approach:

(yield E)      (or just (yield))

with the parentheses in the Pythonic approach not required when the  
yield expression is either (a) the entire expression in an expression  
statement; (b) the right-hand side of an assignment.

Python differs from JS by having only assignment statements, not  
assignment expressions, which simplifies the problem a bit, as the  
grep output cited above shows.

In any case, yield E; as a statement is the most common kind of yield  
in real-world Python code, both because it came first and because  
sending values to generators is less commonly done than iterating  

So with either approach you have some "insignificant, silly  
parentheses" -- but with Neil's approach you have "lots" more because  
the expression-statement yield must have at least () after the keyord.

This tips the usability contest in favor of the Pythonic approach.

In no case are there ambiguities for a bottom up grammar. The right  
parenthesis avoids nasty automatic semicolon insertion problems that  
plagued the unbracketed low-precedence expression on the right of a  
high-precedence prefix in the case of expression-bodied functions, AKA  
"expression closures" (see https://mail.mozilla.org/pipermail/es-discuss/2008-October/007888.html) 

To confirm this, I patched WebKit's Bison grammar for JS to support  
yield //a la// JS1.7. I didn't hook up the semantic actions, just  
verified zero reduce-reduce conflicts. Patch attached below --  
comments welcome (including from you WebKit lurkers :-).


P.S. Bean-counting productions in a bottom-up grammar is counting the  
wrong cost, given the few implementors and their greater skills,  
compared to the many users who need to be spared those painful and  
fruitless parentheses.

Bean-counting productions in a traditional bottom-up grammar is also  
silly given how many are required for ES1-3. From the patch, notice  
all the existing specialized NoIn/NoBF/NoNode variants in parser/ 
Grammar.y (NoBF = No Brace at Front, as far as I can tell).

Much of that production-forking could be avoided by using either  
parameterized LR parsing (www.cs.lth.se/Research/LDTA2004/d09_TheimannNeubauer.pdf) 
, or else top-down ("ad-hoc" or not :-P) parsing.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch
Type: application/octet-stream
Size: 8785 bytes
Desc: not available
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20090517/eb8f4608/attachment-0001.obj>
-------------- next part --------------

More information about the es-discuss mailing list