\u0085 whitespace or a random unicode character by ES5?

Allen Wirfs-Brock allen at wirfs-brock.com
Wed Jul 6 11:54:04 PDT 2011


On Jul 6, 2011, at 11:09 AM, Dave Fugate wrote:

> 
> 
> -----Original Message-----
> From: Peter van der Zee [mailto:ecma at qfox.nl] 
> Sent: Wednesday, July 06, 2011 10:57 AM
> To: Dave Fugate
> Cc: es-discuss at mozilla.org
> Subject: Re: \u0085 whitespace or a random unicode character by ES5?
> 
> On Wed, Jul 6, 2011 at 7:38 PM, Dave Fugate <dfugate at microsoft.com> wrote:
>> Several test262 test cases operate on the assumption '\u0085', aka 
>> Next Line, is considered a whitespace character and I'd like to get 
>> some clarification on whether it really is or not as-per ES5.1.
>> 
>> Table 3 of ES5, Line Terminator Characters, does not call out \u0085 
>> as being a valid line terminator.  It does however state:
>> 
>>                 Only the characters in Table 3 are treated as line 
>> terminators. Other new line or line breaking characters are treated as 
>> white space but not as line terminators.
>> 
>> This raises two questions:
>> 
>> 1.       Is Next Line considered to be a 'new line' or 'line breaking 
>> character'?  By definition, the answer seems to be yes
> 
> I would say no. 7.3 clearly states it's NOT a line terminator.
> 
> [DWF] Sorry, for the confusion - I should have been more clear here.  What I meant was is Next Line considered to be a 'new line' or 'line breaking character' *by Unicode 3.0* (as it's already established ES5.1 does not view it as a line terminator).

I don't think the phrase "new line or line breaking characters" in 7.3 has any normative meaning other than in the context of Table 2- Whitespace Characters. Table 2 only lists category Zs as a source of additional whitespace characters. 0085 is in category Cc so it isn't whitespace.


> 
>> 2.       Next Line is not called out anywhere in Table 2, Whitespace 
>> Characters.  Does this mean Table 2 is simply missing a row for the 
>> clause from "Line Terminators" above, or that the clause should not even exist?
> 
> Or 7.2 should be extended with noting that any unicode line terminator that's not listed in 7.3 is also considered (regular) white space, as per 7.2.
> [DWF] Good idea.

What's a "line terminator"?  Unicode category Zl (line separator, line) includes 2028 and 2029 which are already in table 3. 

There are many other unicode character that contribute to line breaking.  See http://unicode.org/reports/tr14/ 

ES5.1 is precise enough WRT which characters are whitespace and which are Line Terminators.  To me, there seems like only two relevant questions:

1) is there a legacy browser de facto standard of including other Unicode characters in either of these ES token categories.
2) are these real use cases for other Unicode characters into either of these categories.

Given that Dave is asking, I infer that IE does not recognize 0085,  If that has always been the case (ie, prior to IE9) then it seems unlikely that recognition of 0085 has been a defacto standard.

Allen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20110706/d7307946/attachment.html>


More information about the es-discuss mailing list