How can a lexer decide a token to be "get", "IdentifierName" or "Identifier" ?

[Michael Dyck]

程劭非 wrote:
> Though it's a little too long since this discussion, I've tried Allen's 
> idea in my parser and still find conflicting.
> 
> Consider the following rules:
> 
> PropertyAssignment :
>     IdentifierName PropertyName ( ) { FunctionBody }
> 
> PropertyAssignment :
>     PropertyName : AssignmentExpression
> 
> PropertyName :
>     IdentifierName
> 
> when a parser get “IdentifierName” it need to decide reduce 
> the IdentifierName into "get" or “PropertyName”.

Strictly speaking, it wouldn't reduce IdentifierName to "get", because 
there isn't a production
     "get" : IdentifierName
Instead, it's a shift-reduce conflict.

> For LR parsers there is no way to do these things.

There's no way only if you're talking about an LR(0) parser. But an LR(1) 
parser would have 1 token of lookahead to resolve the conflict:
  -- if the next token is ":", reduce IdentifierName to PropertyName;
  -- if the next token is an IdentifierName, shift that.
  -- if the next token is anything else, syntax error.

Similarly, an LL(1) parser wouldn't be able to decide between the two 
alternatives for PropertyAssignment, but an LL(2) could do it.

(All of this is ignoring the effects of other productions.)

-Michael