complexity tax

[Mark S. Miller]

On Tue, May 20, 2008 at 7:02 AM, Mark S. Miller <erights at google.com> wrote:
> 2008/5/19 Brendan Eich <brendan at mozilla.org>:
>> On May 19, 2008, at 6:54 PM, Mark Miller wrote:
>>
>>> If I could
>>> have a language with some of the syntactic conveniences of ES4 but
>>> without ES4's semantics, I'd be quite happy.
>>
>> What semantics in particular, can you pick on something specific that's not
>> in your classes as sugar proposal (rather than have me guess)?
>
> Good suggestion. Will do. Top of my list will be first-class Name objects.


I have now read the three draft ES4 spec docs Lars sent out. Yes, the
tops of my list are names and namespaces. The complexity it adds
pushes the property lookup resolution algorithm way beyond what
programmers, including myself, can reliably predict. (Grain of salt
alert: I had the same reaction to C++'s overload resolution rules and
their interaction with C++'s implicit coercion rules. Nevertheless,
C++ is widely used. Draw from this what conclusions you will.)

AFAICT, there are three use-cases for namespaces in ES4:

a) At the package-ish level, in order to have some discipline over
inter-module-ish linkage and who gets to import/export what global-ish
variable names to whom.

b) At the property-name level, in order to be able to add additional
properties to existing objects without breaking existing code that is
already using those names for other reasons.

c) As a secure encapsulation mechanism, so that the impossibility of
mentioning some property names prevents access to the properties
associated with those names.


I don't find any of these use cases compelling. Out of order:


a) The YUI and Dojo libraries (and perhaps others) already adopts a
convention of dotted path names for namespace discipline of global-ish
variables and inter-module-ish linkage. This has the familiar look of
Java-ish package names and fully qualified class names, and is built
as a pattern, with no appreciable inconvenience, out of existing ES3
concepts.


c) As an encapsulation mechanism, this provides implicit sibling
access, as in Java. Java "private" is per-class. Joe-E inherits this
design from Java, and uses it successfully for security. Any instance
of a Joe-E class can implicitly see the private members of any other
instance of that class. By contrast, Smalltalk's instance variables
are encapsulated per-instance. E and Caja follow Smalltalk, and use
this design successfully for security.

We have discussed on cap-talk and/or e-lang the confused deputy
hazards of Joe-E's per-class encapsulation as contrasted with E's
per-instance encapsulation. The problem is that the greater access
siblings have to each other -- beyond what clients have to each --
provides an implicit form of rights amplification. (See section 10.2.1
and Figure 10.1 of <http://www.evoluware.eu/fsp_thesis.pdf> for how
rights-amplification raises confused deputy hazards.) Our conclusion
was that this is indeed a hazard, but a lint-like tool for Joe-E could
mitigate the danger since such amplification is always statically
detectable. (Access to a non-public member of something other than
"this".)

ES4 has no such reliable ability to detect amplification statically.
Therefore, the confused deputy hazards inherit in using rights
amplifications will be non-obvious and difficult to detect. Better to
have an attribute-based encapsulation mechanism, that explicitly
distinguishes public vs private properties, and to use per-instance
encapsulation of private properties. (Adding a "private" attribute to
properties would be much more elegant that Caja's current "does it end
with an underbar?" test.) When rights amplification is actually
needed, better to express it explicitly by other means, such as the
sealer/unsealer pairs shown in the Caja spec.


b) Having dispensed with the utility of ES4 namespaces as a secure
encapsulation mechanism, there remains the issue of using them by
convention/discipline to avoid accidental collisions. Experience with
Smalltalk and the Prototype library for JS shows that there really is
a genuine problem here. In Smalltalk, hundreds of separately written
extensions add methods to Object. Early JavaScript libraries like
Prototype did likewise. Existing Smalltalk virtual machines, going
back at least to Smalltalk-80, already support method dispatch on
non-string selectors, so an ES4-like solution was already possible
with only a change to the compiler. I know of at least three designs
(the best is <http://home.cc.gatech.edu/tony/uploads/32/SharedSmalltalk.htm>)
to bring this power to Smalltalk. The T language (an object-oriented
variant of Scheme) and Joule (E's immediate ancestor) all had some
form of first-class selector
<http://www.erights.org/history/joule/MANUAL.B17.pdf>.  In all of
these, as in ES4, a source identifier is first looked up in a lexical
context to find a first-class selector object. Then, this selector
object is used for method-dispatch (i.e., property lookup).

None of these caught on. Adoption is a complex process, so lack of
adoption is not strong evidence of anything. However, some interesting
data:

* Java classes are not extensible the way Smalltalk's classes are. In
Java, I have to put up with the pain that my module is unable to
extend Object. In exchange, I gain the benefit that your module is
also unable to extend Object. Talking to several programmers who made
the shift from Smalltalk to Java (not a scientific survey, and I'm
sure I asked leading questions), the reports I got were that the gain
was worth the pain.

* More recent JavaScript libraries have learned to avoid Prototype's
practice of extending primordial objects. Rather, by convention and
discipline, they largely program as if subject to Java's
non-extensibility constraints. Extending primordial objects is now
widely regarded as bad practice among JavaScript programmers.

* E descends directly from Joule but dropped this aspect of Joule.
Although we keep expecting to miss it, we haven't actually missed it
in practice. E objects are not extensible by their clients -- not
actually, and not virtually.



So, although I don't find any of these use-cases compelling, I'd agree
that #b is at least a real problem, and is the most plausible of the
arguments for some additional support in a future JavaScript. My next
message, if I get to it, will be "Namespaces as Sugar". If you agree
with me that #b is better avoided than solved, feel free to skip it.


-- 
    Cheers,
    --MarkM