Directive prologue members, escapes, and retroactive syntax errors

Peter van der Zee ecma at qfox.nl
Sat Oct 16 02:33:10 PDT 2010


> So you suggest the tokenizer should have two modes of operation,
> one to get a token using strict mode parsing and one to get it
> using normal parsing?  

Doesn't the tokenizer already have this mode? How else is it able to know
whether it should parse an octal in strings? And which additional keywords are
additionally restricted in strict mode? You have to explicitly switch cfg's
because of the second clause in 7.6.1.2.

In addition to this the tokenizer would have to know that it is in the "header"
of a Program (whether it be the entire script or a function body). As long as it
is, it could parse strings in a generic way. This should not have any side
effects, so as long as they are proper string literals and not part of a more
complex expression.

Actually, the errata you (Jeff) proposed at the beginning of this thread,
disallowing escapes in directives, might sound like the easiest way out after
all...

I would still like to see additional productions for a DirectivePrologue,
including whatever restrictions might be agreed on...

Brendan: well for my own parser the lookhead is kind of "free" (or inherent
because it always does an exhaustive tree search). In the production I gave
earlier, the lookahead cost would be at most one StringLiteral, since as soon as
the next character after that string is not a semi-colon or newline, you know
you're no longer parsing a directive. However, that's still cheaper than rolling
back multiple strings that might preceed a strict directive.

- peter


More information about the es5-discuss mailing list