optional "function" keyword

Kevin Smith khs4473 at gmail.com
Thu Mar 8 19:35:53 PST 2012

I think we can use a modification of the Dart technique to parse "=>"

On encountering "(" in a context where an expression is acceptable, you can
try to parse it as ( Expression ) or as ( FormalParameterList ).

Attempt to parse it as a formal parameter list, enqueuing each token
encountered.  Either the parameter list will be successfully parsed or not.

If a parameter list is successfully parsed, then

  peek at the next token to see if it's a "=>".  If it is, then
  clear the queued tokens and continue parsing as normal.
  If it's not a "=>", then put the queued tokens back into the token
  stream and restart parsing from the original "(", this time as
  an expression.

otherwise, if we fail to parse the parameter list, then

  put the queued tokens back into the token stream and restart
  parsing from the original "(", this time as an expression.

Lemma:  The "lookahead" token stream that we queue while attempting to
parse ( FormalParameterList ) will be identical to the token stream that we
would have queued if we had attempted to parse ( Expression ).


On Thu, Mar 8, 2012 at 4:28 PM, Kevin Smith <khs4473 at gmail.com> wrote:

> Thanks for the clear explanation, Brendan - I think I understand the
> concerns now.  And if not, I don't mind looking foolish : )
> I just looked at the code for the Dart parser and it basically queues
> tokens until it finds the matching right paren (recursively counting nested
> parens).  It peeks at what comes afterward (like a => or {), and then it
> goes back to the original left paren and resumes parsing, dequeueing the
> already scanned tokens.
> Would this technique work for us?  The problem that I see is that in JS,
> you can't really tokenize without also parsing (because of the cursed
> lexical ambiguity on '/').
> kevin
> On Thu, Mar 8, 2012 at 2:35 PM, John Tamplin <jat at google.com> wrote:
>> On Thu, Mar 8, 2012 at 1:08 PM, Brendan Eich <brendan at mozilla.org> wrote:
>>> Another which I cited just a few messages back: parsing ( params ) as (
>>> expr ), as any top down parser must until it gets to an arrow or other
>>> distinguishing token ({ on same line as ), e.g.), can be considered
>>> future-hostile. params should be declarative, but expr must now cover all
>>> future declarative syntax, including optional guards.
>> For this particular problem, the Dart parser looks ahead to see if it
>> could be a function declaration.  If so, it parses it that way, otherwise
>> it parses as a parenthesized expression.  I don't know if this sort of
>> approach would be considered acceptable here or not (it does impose some
>> restrictions on the scanner for n-token lookahead).
>> --
>> John A. Tamplin
>> Software Engineer (GWT), Google
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20120308/08e4e2a5/attachment.html>

More information about the es-discuss mailing list