Opt-out schemes for ECMAScript 3.1

Brendan Eich brendan at mozilla.org
Fri Nov 28 00:33:23 PST 2008


On Nov 18, 2008, at 8:34 PM, William Edney wrote:

> All -
>
> In reviewing the proposed ECMAScript 3.1 specification, we have  
> identified what we feel is a grave situation for the ECMAScript  
> community and the Web at large: the omission of an explicit opt-in  
> mechanism for ECMAScript 3.1.

The last work on opt-in versioning was recorded here:

http://wiki.ecmascript.org/doku.php?id=proposals:versioning

For ES3.1 we hope to extend existing objects with new properties not  
used by common libraries, or else by codifying existing library  
practice. More below in detail.


> These issues have been debated elsewhere and a variety of counter- 
> arguments have been put forth. However, in our view:
>
> 1. It is insufficient to claim that 'these changes are minor'.

Agreed.


> This is irrelevant since they overlay existing semantics, no matter  
> how 'minor' they are. They have the potential to break the Web, and  
> that's not 'minor'. For example, when 'foreach' is defined on  
> Array.prototype by a JS library, Mozilla engines implementing  
> JavaScript 1.6 hang.

I'm guessing you mean forEach -- but I am unable to reproduce. Can you  
mail me a testcase or just file a bug? Thanks.


> 2. It is insufficient to claim that 'object testing can be performed  
> to work around new features that are unwanted'. Again, this is not  
> relevant. Scripts in the field authored years ago by a long-gone  
> developer cannot be magically upgraded via time machine to do object  
> testing for features that are now being standardized or implemented  
> by a browser vendor or specification committee.

Agreed again. The hope is to match the de-facto standards where they  
exist.


> 3. It is insufficient to claim that 'if you define these properties,  
> they'll just overlay ones that we're defining for ES 3.1'. Many of  
> these same-named properties have different semantics that vary from  
> library to library.

Let's get to specifics to make progress here.


> What about new ES 3.1 engine code that is calling this new  
> functionality and is now dealing with a JS-defined property that  
> behaves very differently?

ES3.1 generally avoids depending on overrides. This is a reality-based  
(i.e., implementation-driven) correction to ES1-3, particularly for  
use of standard constructors and prototypes. See:

http://wiki.ecmascript.org/doku.php?id=clarification:which_prototype

and the Kona draft's use of "standard built-in  
constructor" (redundancies aside, as DSH noted in the "new Object()"  
thread just posted today).


> This is related to point #1. Many libraries out there add a 'bind'  
> property to Function.prototype, and at least the ones we know of  
> don't take a varargs list for their version of 'bind'.

Where would a bind override be used by ES3.1 pseudo-code or prose?

I'm skipping your point 4. I agree with it in general, but the  
specifics matter.

ES3.1 has limited scope in which to dodge the opt-in versioning  
bullet. Let's address its specific compatibility hazards in order to  
avoid over-engineering versioning now for 3.1, instead of in Harmony  
when we have more time to spec and trial-implement, and get it right.


> 5. It is insufficient to claim that 'well, we've surveyed a lot of  
> code out there and so far haven't found any conflicts with what  
> we're proposing'. There's a lot of JavaScript that doesn't exist on  
> the Internet. It exists in non-public web sites by the truckload and  
> is not visible to us. This community is not omniscient around how  
> JavaScript is being used 'out in the wild'.

Absolutely agree, which is why TC39 has agreed to two interoperating  
implementations available for testing before the specification is  
finalized. Of course, this is not as good as shipping those to mega- 
user cohorts and waiting years for complaints, but we can't afford to  
do that -- we'll unwind and implementations will go their own ways, or  
something worse will happen.


> isCallable operator
>   New operator checks for a call method and will return inconsistent  
> results across browsers (RegExp in some implementations will return  
> true but not in others since RegExp.prototype.call isn't part of the  
> standard).

There is no RegExp.prototype.call in Mozilla's implementations, but I  
see your point. It has led us to revert typeof /hi/ to evaluate to  
"object" in Firefox 3, and we're likely to get rid of callable regexps  
soon too, as I've written here.

But this is not an ES3.1 issue. It's a Mozilla issue.


>   Conflicting framework implementation accepts any object and  
> returns true if the object is a) a Function instance, b) not marked  
> as a DNU.

(DNU = doesNotUnderstand for those not SmallTalk-aware.)

I don't see the conflict. IsCallable is an internal method in ES3.1.  
It's not exposed to script and existing "callable" predicates should  
do what they've always done. Any disagreement will not surprise old  
code, since it won't use new forms that depend on the new internal  
method. Again the callable regexp bug is Mozilla's, not Ecma's.


> 10.0 Executable Code and Execution Contexts:
>   What is the characterization of code which is added to a program  
> dynamically by adding a <script>inline_code_here</script> tag? Is  
> that considered "eval code" or are the separate contents treated as  
> global, function, and eval code respectively?

This question has nothing to do with ES3.1 either. Since ES1, script  
tag inline content and out-of-line src are both "global code".


>   As a related question, what is the strict lexical context of such  
> code? (currently it would appear to be the global context which  
> seems appropriate)

Lexical != dynamic, so strict is a property of each script, not a  
sticky property of a global object.


> 10.1.1 Strict Mode Code
>   Is the intent here that different <script> nodes would be able to  
> define different settings for strict vs. unrestricted?

Yes.


> 11.1.5
>   In the syntax discussion on PropertyAssignment the verbs "get" and  
> "set" are used. Are these going to be treated like keywords?  
> Reserved words? Something else? They will likely conflict heavily  
> with frameworks which already use get/set "bean-like" APIs for  
> encapsulating property access.
>   As an aside, getters can take parameters too, at least in the  
> environments I've worked with in the past, so not supporting a  
> parameter list for getters might restrict their value in those more  
> complex cases.

This object initialiser syntax is all based on Mozilla's  
implementations, cloned in 3 of 4 other top browsers. The get and set  
identifiers are not new unconditionally-reserved words, they're  
keywords in context. They are part of new (to ES3) syntax, which  
cannot collide with existing library code built on prior syntax.


> 12.1 Block Scope
>   Please clarify, it appears this will continue to work for existing  
> code due to how lexical contexts are mapped. Is that correct? In  
> other words, will the current functionally-scoped vars etc be  
> visible to the block? Presumably yes.

Yes, at last week's meeting we fixed strict mode to allow var in block  
statements and other cases the Kona draft was trying to forbid. Look  
for the obvious fixes (removals) in a new draft.


> 15.2.3.5 Object.create
>   Conflicts with frameworks which have already implemented this  
> method as an alternative constructor to using raw "new X" calls for  
> object creation. In particular, create is overridden by subtypes and  
> performs differing operations depending on the type.

What frameworks? Is it really a method on Object?

Prototype has Class.create, but that does not conflict.


> 15.2.3.14 Object.keys
>   Conflicts with frameworks which have already implemented keys  
> properties, particularly for custom Hash object types.

Prototype has an Object.keys. My position at Kona was that we should  
standardize that: for-in order and no optional "fast" param. TC39  
members in attendance agreed. Is there any other precedent here? If so  
does it conflict with Prototype?


> 15.3.4.5 Function.prototype.bind
>   Almost certain to conflict with all major AJAX frrameworks, since  
> they almost all implement a bind method whose signature and  
> semantics are somewhat different.

I'll defer getting into the details for now and argue that we are  
trying to standardize a common bind that works like these precedents.  
If they are not all compatible with some useful union or intersection,  
we'll have to consider dropping back to perhaps less useful but common  
intersection semantics, or (I hope not!) using another name.

For now we are going to try to unify and heal -- it looks doable, but  
we'll find out before standardizing. Please cite known conflicts and  
hard cases.

At the Kona meeting I pointed out that ES3.1's sharing of .prototype  
between the |this| function and its partially applied form returned  
by .bind (in order to support operator new) was incompatible with  
existing libraries such as Prototype. We agreed to field-test this and  
see what breaks. I've never seen a .bind result new'ed, but I welcome  
evidence that this is done in ways that shared .prototype would break.

Yes, we are on the slippery slope, but I think we should take this  
chance. A bind method is insanely common, yet tedious to reinvent or  
acquire. The ones I've studied have a lot in common. The common cases  
should be standardized. You're right that doing so might require opt- 
in, but that's not for sure yet. We still have time to rename or add  
opt-in or opt-out, as we learn more in the next few months.


> 15.3.5.4 Function.prototype.name
>   This will conflict with at least one framework which currently  
> sets a name property on Function instances to store their name, and  
> which assigns 'anonymous' as the name when not found.

This refers to:

15.3.5.4 name
The value of the name property is the name of the function, or an  
empty string if the function is anonymous. This property has the  
attributes { [[Writable]]: false, [[Enumerable]]: true,  
[[Configurable]]: false }.

I missed this, thanks for pointing it ou. It is incompatible.  
SpiderMonkey has a name property of Function instances (note: not  
Function.prototype.name), but it's read/write. This is a good catch.

I'll get the trac opened up next week and log this one.


> 15.4.3.2 Array.isArray
>   Admittedly this is a "type method" rather than an instance method,  
> but the isArray function has historically returned true if the  
> argument was a Java array (due to LiveConnect) so this would change  
> the semantics depending on the context.

You're now arguing about what you would prefer for semantics of a new  
ES3.1 method. You are not citing any evidence of a compatibility  
problem hinging on the name Array.isArray.

On the semantic point, we need a way to test [[Class]] == "Array", in  
the spec's own terms and for self-hosting things like  
Array.prototype.concat. Anything catering to LiveConnect can be  
layered on top by some other spec or implementation.


> Date.now
>   Collision

With what? We've shipped this static method for many years. I know of  
no conflicting precedent.

This is another case of standardizing /de jure/ what has been de-facto  
standardized by implementations.


> *.prototype.toJSON
>   Collision

IIRC, the precedent is crock's json2.js. Is there any conflict?


> 15.5.4.20 String.prototype.trim
>   Collision

What specific precedent do you mean, and how does the 3.1 method differ?


> Additional Array iterators.
>
> 15.4.4.15 Array.prototype.lastIndexOf
> 15.4.4.14 Array.prototype.indexOf
>   Will conflict with frameworks already implementing this function.  
> The semantics do appear to be consistent with those of the  
> implementations I'm aware of though, so this probably won't cause  
> failures.

These have been in SpiderMonkey for years, and they were cloned along  
with the map/filter/etc. methods by other engines (I believe -- check   
my claim here, please).


> 15.12 JSON
>   Conflicts with frameworks which have already implemented this  
> object.

See above about json2.js, which is pretty old.


> Annex D
>   Changes to RegExp.prototype will cause isRegExp() in at least one  
> framework to break since it relies on the fact that RegExp objects  
> currently return Object as a way of distinguishing them from  
> Function instances which also implement both apply and call in  
> certain browsers (thanks to prior releases of new functionality).

No, changing RegExp.prototype to have [[Class]] "RegExp" does not  
affect typeof (what you must mean by "current return Object").

The Mozilla callable RegExp kerfuffle is our (Mozilla's) bug to fix.

This Annex D informative item is a change to match what all browsers  
(or 3 of 4, I forget) do. ES3 tried to say that only RegExp.prototype  
was an Object instance, while all other native prototypes (e.g.  
Date.prototype) were first-born instances of their constructors. It's  
de-facto => de-jure standardization again, a good thing.



> If future versions continue to define properties with no explicit  
> opt-in / opt-out mechanisms in place, the consequences get more  
> serious with each passing year as JavaScript's popularity curve is  
> increasing, not decreasing. There may be an ES 3.2... who knows?

Not likely because the spec formalism must die. Harmony or bust.


> For that reason we strongly urge the committee to consider using the  
> same opt-in approach for ES 3.1 that will be required for ES Harmony.

That's plausible, but in my experience, most users won't version- 
select carefully.


> Failure to require an opt-in approach with the standard _will_ break  
> existing scripts.

We'll find out, but we aim to codify existing practice. This is easier  
in Python or Perl with explicit deprecation/obsolescence machinery and  
big-stick implementation leverage (only one C-Python, people want to  
use its new features). It's hard on the web, but still doable. The ES2  
and ES3 standards codified existing practice too.

Try looking at it the other way 'round: developers hate MIME types and  
forget to use them even if they believe they ought to. It's likely  
that 3.1 code will leak into default version space. We will test- 
implement as we finish the 3.1 spec and try to find breaking changes.  
If we can't fix them (somehow), we'll have to rename or remove them.


> Our proposed opt-out approach would take the following form:
>
> An addendum to the specification whereby ES engine vendors agree to  
> use a host-language mechanism that will cause their engines to shift  
> into 'ES 3.0 only' mode, ignoring all of the new properties and  
> features of ES 3.1.

We have some code to do this in SpiderMonkey. This kind of "opt- 
in" (it's not really opt-out -- nothing "opts" by defaulting to the  
unspecified version) is necessary for reserved identifiers, even  
though they are contextually reserved only (see http://wiki.ecmascript.org/doku.php?id=proposals:reserved_words) 
.

It's possible we'll have to extend this version selection machinery to  
handle JSON, but I doubt we will. IE8 is shipping ES3.1 JSON already,  
and this smoked out a facebook .js file that conflicted with json2.js  
and ES3.1. I believe it has been fixed (I'd like to confirm that  
kill ;-).

Anyway, your thinking here is good but we are not ready to foist this  
versioning complexity on the ES3.1 spec, which bulks up the tired old  
spec-formalism bones of ES1-3.


> Here is an example of what we would propose:
>
> <script type="application/javascript;version=1.5">...</script>
>
> A second form (for those in the pure ECMAScript world):
>
> <script type="application/ecmascript;version=3.0">...</script>

See RFC 4329 (http://www.ietf.org/rfc/rfc4329.txt). This is covered to  
some extend by

http://wiki.ecmascript.org/doku.php?id=proposals:versioning

but not incorporated into ES3.1.


> Note that this has the distinct advantage of being able to be used  
> by markup developers with no ECMAScript development experience who  
> can 'tweak their script tag to make their scripts run again'.

Or not -- Larry Masinter points to MIME type botches common on the web  
and in server config files. He's right; XML namespace URIs are as bad  
or worse. Good thing the client dictates inline and src script type  
and we don't have to worry about server configs!

But you are still creating a huge fat-finger hazard here, in expecting  
much developer opt-in for what should be a codify-existing-practice ". 
1" upgrade to ES3.

I'll stop here, because I continue to believe that we can codify  
existing practice in standardizing ECMAScript. I'm also skeptical of  
opt-in versioning's usability. It therefore seems to me more helpful  
at this juncture to study details about known and potential problems  
(conflicts among existing libraries for common names newly specified  
in ES3.1), than to jump with both feet to a 3.1 versioning solution.

/be


More information about the Es-discuss mailing list