Full Unicode strings strawman
bzbarsky at MIT.EDU
Mon May 16 14:42:51 PDT 2011
On 5/16/11 4:38 PM, Wes Garland wrote:
> Two great things about strings composed of Unicode code points:
> If though this is a breaking change from ES-5, I support it
> whole-heartedly.... but I expect breakage to be very limited. Provided
> that the implementation does not restrict the storage of reserved code
> points (D800-DF00)
Those aren't code points at all. They're just not Unicode.
If you allow storage of such, then you're allowing mixing Unicode
strings and "something else" (whatever the something else is), with bad
most likely bad results.
Most simply, assignign a DOMString containing surrogates to a JS string
should collapse the surrogate pairs into the corresponding codepoint if
JS strings really contain codepoints...
The only way to make this work is if either DOMString is redefined or
DOMString and full Unicode strings are different kinds of objects.
> Users doing surrogate pair decomposition will probably find that their code "just works"
> Users creating Strings with surrogate pairs will need to
Such users would include the DOM, right?
> but this is a small burden and these users will be at the upper
> strata of Unicode-foodom.
You're talking every single web developer here. Or at least every
single web developer who wants to work with Devanagari text.
> I suspect that 99.99% of users will find that
> this change will fix bugs in their code when dealing with non-BMP
Not unless DOMString is changed or the interaction between the two very
carefully defined in failure-proof ways.
> Why do we care about the UTF-16 representation of particular
Because of DOMString's use of UTF-16, at least (forced on it by the fact
that that's what ES used to do, but here we are).
> Mike Samuel, can you explain why you are en/decoding UTF-16 when
> round-tripping through the DOM? Does the DOM specify UTF-16 encoding?
> If it does, that's silly.
It needed to specify _something_, and UTF-16 was the thing that was
compatible with how scripts work in ES. Not to mention the Java legacy
if the DOM...
> Both ES and DOM should specify "Unicode" and let the data interchange format be an implementation detail.
That's fine if _both_ are changed. Changing just one without the other
would just cause problems.
> It is an unfortunate accident of history that UTF-16 surrogate pairs leak their
> abstraction into ES Strings, and I believe it is high time we fixed that.
If you can do that without breaking web pages, great. If not, then we
need to talk. ;)
More information about the es-discuss