\u0085 whitespace or a random unicode character by ES5?

Allen Wirfs-Brock allen at wirfs-brock.com
Wed Jul 6 15:35:11 PDT 2011


On Jul 6, 2011, at 3:09 PM, Dave Fugate wrote:

> Hi Allen, this is a case where Canary and IE9 agree that \u0085 is whitespace or a line terminator as far as String.prototype.trim is concerned, but other browsers do not.  I’ll file a spec bug to clarify the text as there seems to be confusion on the Web about this one

I don't think there is any ambiguity in the spec about trim. It references the WhiteSpace and LineTerminator grammar productions which are quite precise i their definition. Since trim is new with ES5 there presumably aren't legacy compact issue in regard to it.  It looks to me that Canary and IE9 are just just out of spec WRT trim.

> (e.g., do a search on “ECMAScript” within http://en.wikipedia.org/wiki/Newline). 

I saw that, and as far as I can tell its assertion about ECMAScript and U+0085 is just wrong. I found no evidence that ES ever specified U+0085 to be a line terminator.  Of course, various implementations may have.

> Also, doesn’t help that Unicode 3.0 calls out \u0085 as being a line break.

The ES spec. defines ES tokens, not the Unicode spec.   Also, the Unicode concert of line breaks is quite complex as I mentioned earlier see Unicode TR14.

Allen

>  
> Any ways, thanks for everyone’s help!
>  
> Dave
>  
> From: Allen Wirfs-Brock [mailto:allen at wirfs-brock.com] 
> Sent: Wednesday, July 06, 2011 11:54 AM
> To: Dave Fugate
> Cc: Peter van der Zee; es-discuss at mozilla.org
> Subject: Re: \u0085 whitespace or a random unicode character by ES5?
>  
>  
> On Jul 6, 2011, at 11:09 AM, Dave Fugate wrote:
> 
> 
> 
> 
> -----Original Message-----
> From: Peter van der Zee [mailto:ecma at qfox.nl] 
> Sent: Wednesday, July 06, 2011 10:57 AM
> To: Dave Fugate
> Cc: es-discuss at mozilla.org
> Subject: Re: \u0085 whitespace or a random unicode character by ES5?
> 
> On Wed, Jul 6, 2011 at 7:38 PM, Dave Fugate <dfugate at microsoft.com> wrote:
> 
> Several test262 test cases operate on the assumption '\u0085', aka
> Next Line, is considered a whitespace character and I'd like to get
> some clarification on whether it really is or not as-per ES5.1.
>  
> Table 3 of ES5, Line Terminator Characters, does not call out \u0085
> as being a valid line terminator.  It does however state:
>  
>                 Only the characters in Table 3 are treated as line
> terminators. Other new line or line breaking characters are treated as
> white space but not as line terminators.
>  
> This raises two questions:
>  
> 1.       Is Next Line considered to be a 'new line' or 'line breaking
> character'?  By definition, the answer seems to be yes
> 
> I would say no. 7.3 clearly states it's NOT a line terminator.
> 
> [DWF] Sorry, for the confusion - I should have been more clear here.  What I meant was is Next Line considered to be a 'new line' or 'line breaking character' *by Unicode 3.0* (as it's already established ES5.1 does not view it as a line terminator).
>  
> I don't think the phrase "new line or line breaking characters" in 7.3 has any normative meaning other than in the context of Table 2- Whitespace Characters. Table 2 only lists category Zs as a source of additional whitespace characters. 0085 is in category Cc so it isn't whitespace.
>  
> 
> 
> 
> 
> 2.       Next Line is not called out anywhere in Table 2, Whitespace
> Characters.  Does this mean Table 2 is simply missing a row for the
> clause from "Line Terminators" above, or that the clause should not even exist?
> 
> Or 7.2 should be extended with noting that any unicode line terminator that's not listed in 7.3 is also considered (regular) white space, as per 7.2.
> [DWF] Good idea.
>  
> What's a "line terminator"?  Unicode category Zl (line separator, line) includes 2028 and 2029 which are already in table 3. 
>  
> There are many other unicode character that contribute to line breaking.  See http://unicode.org/reports/tr14/ 
>  
> ES5.1 is precise enough WRT which characters are whitespace and which are Line Terminators.  To me, there seems like only two relevant questions:
>  
> 1) is there a legacy browser de facto standard of including other Unicode characters in either of these ES token categories.
> 2) are these real use cases for other Unicode characters into either of these categories.
>  
> Given that Dave is asking, I infer that IE does not recognize 0085,  If that has always been the case (ie, prior to IE9) then it seems unlikely that recognition of 0085 has been a defacto standard.
>  
> Allen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20110706/92162896/attachment-0001.html>


More information about the es-discuss mailing list