Module naming and declarations

Sam Tobin-Hochstadt samth at ccs.neu.edu
Tue Apr 30 16:15:31 PDT 2013


On Mon, Apr 29, 2013 at 9:34 AM, Andreas Rossberg <rossberg at google.com> wrote:
> On 26 April 2013 00:27, Sam Tobin-Hochstadt <samth at ccs.neu.edu> wrote:

Dave has already responded about URIs, URLs, and external names, so
I'll focus on lexical modules and module declarations below.  This
includes response to a couple other of your emails inline.

>> #### The key takeaway
>>
>> The key takeaway here is this: in the current design, different
>> modules from different developers can coordinate based on the
>> *internal* names of their dependencies. In your lexical proposal, it's
>> not possible to coordinate globally on internal names, because they're
>> lexical.  So instead developers would have to coordinate on *external*
>> names.  This is fundamentally flawed, because external names are about
>> where to get bits, not which abstraction a name represents.
>
> We seem to have some misunderstanding about the nature of internal
> names. As far as I'm concerned, in both approaches, coordination is
> going through external names. The difference, as far as I can tell, is
> that you seem to suggest abusing the URI mechanism for short logical
> names that violate URI semantics, whereas I am saying that conformant
> integration into the syntactic _and_ semantic structure of URIs is
> vital.

Ok, I think I was misunderstanding your use of terminology. The names
mapping in the registry in the current system I was calling internal
names, and you think of them as external names.

>From now on, I'll stick with "logical name" for a name that goes in
the registry ("jquery/ui") and "URL" for something where the physical
source is retrieved from the internet
("http://jquery.com/modules/jquery/ui.js").

>
>> Supporting this use case properly is what led us to realize that the
>> earlier lexically-named proposal was flawed.
>>
>> Note that none of this is about concatenation.  We've made some
>> particular design decisions where concatenation played a role, but it
>> wasn't a part of the reason we moved away from lexical modules.
>
> Concatenation is the only reason I can see for the non-lexical module
> declarations in the current proposal, and if I remember correctly,
> that was the main motivation you or Dave gave at the last meeting. Is
> that correct?

> I brought up [removing module declarations] as a "min-max" sort of compromise at
> the March meeting, but it did not get much love. And I agree that it
> isn't pretty, although it actually has some technical advantages, e.g.
> enabling parallel parsing.

No, that isn't correct. We've been over this a few times at this
point, but once more into the breach ...

There are several reasons why module declarations are important.

1. Right now, every JS construct can be written at the REPL, in a
script tag, in a bunch of files in a Node package, or anywhere else.
This is a very important property for the language, and one we should
maintain.  Without module declarations except as separate files, we
would lose this uniformity.

2.  Being able to define multiple modules concisely in a single file
is simply more convenient than having to separate them into multiple
files. Even clunky old Java allows this simple convenience!

3. Serving modules as script tags, and defining modules in inline
script tags, is likely to be important for a wide variety of use
cases.

4. Concatenation.

> As for a SPDY-like mechanism, I absolutely believe that something
> along these lines is the way to go, for two reasons:
>
> 1. My prediction is that in the mid-term future, concatenation will
> actually become harmful to performance when the sources are already in
> the browser cache, because it prevents a lot of parallelisation,
> caching and/or work deferring that would otherwise be possible in the
> VM.
>
> 2. The bundling and transmission issue is a problem far beyond JS
> files, and needs to be solved on a more general level. Large web
> applications like Gmail currently go to great length to optimize it
> manually, across file types, and I believe that a purely JS modules
> focused approach to bundling will not help them a iota, whatever we
> do.

I realize that you don't seem to find concatenation an important use
case, but I think this is wrong. For the medium-term, http requests
are expensive, and concatenation is a reality.  Browsers haven't made
extra requests free, let alone cheaper, and it would be a big mistake
to build a system that assumes this.

Large web apps use concatenation as part of their approach to
performance today, despite its limitations.

[From another email]
> I do think lexical names are essential to provide internal naming,
> which is vital. Thus having lexical modules was the first half of my
> suggestion. "Public" modules however are externally visible, so are
> naturally named by external names. The second half of my suggestion
> was that we should use proper URIs for that. The "third half" was
> about properly separating these concerns.

Using your terminology, the current design uses names like "jquery/ui"
for internal naming.  You're suggesting that we really need another
level of internal naming, based on lexically-scoped identifiers.

This would be a coherent addition to the system, but is not a
necessary one. Below, you point out some issues that would be
potentially addressed by adding them.  However, multiple JS module
systems are in use today, and with the partial exception of the
TypeScript namespaces feature (which is a significantly different
design), none of them has lexically-named modules in the sense you
advocate.

>> #### Some smaller points
>>
>> Also, the incoherent programs you refer to are ruled out by a
>> suggestion (of yours!) that we adopted at the last meeting:
>> declarative module forms that define a module that already exists are
>> a static error.
>
> The suggestion you mention rather deals with the next case (clashes
> between different internally defined modules). It does not generally
> help when embedded resources clash with externally defined resources.

When the second resource loaded is a declarative form, then this
addresses this case as well.

>>> * Likewise, a single global name space for all internally defined
>>> modules can lead to incoherent programs.
>>
>> I would be more worried about this if (a) we didn't provide convenient
>> ways to structure this name space and (b) it wasn't already an
>> existing successful approach in real-world JS systems.
>>
>>> * "Local" references are globally overridable.
>>
>> This is only true in the sense that executing first lets you define
>> the meaning of particular modules.  But this is a *feature*.  This is
>> just another way to describing configuration.  It's not reasonable to
>> think that we can decide for everyone what will need configuration and
>> what won't.
>
> Er, I'm not saying that _we_ should decide it. I'm saying that the
> implementer of a module should be able to make that choice. In the
> current proposal, he doesn't have a choice at all, he is _forced_ to
> make each and every module definition essentially "mutable" by the
> rest of the world.

There are two possible scenarios here. If your module is loaded inside
a loader that you don't trust, you have _zero_ semantic guarantees.
The translate and link hooks give the loader arbitrary power, lexical
modules or no. The initial loader in the JS environment is mutable and
thus if you are in a mutually-untrusting situation, you can't rely on
anything.  Nonetheless, this is the right default for the system we're
designing, and it can be configured and new loaders built for other
situations.

In contrast, if you _can_ trust the loader, and the loader _isn't_
exposed to arbitrary mutation, then you're in a much better position.
After two modules are linked, the link never changes. Linking is a
configurable process, but its *results* are immutable. This is in
contrast to how JS programs that use the top level as a namespace work
today, for example. Thus, two modules in a script tag or file that are
linked together are just as bound as they would be in a system with
lexical modules.

>>> * Internal module definition is coupled with external module
>>> registration, and modules cannot be renamed or re-registered.
>>
>> These are not "external" names, but it is true that declaring a module
>> registers it.  It's possible to take it out of the table afterward, or
>> give it a new name, or have it share multiple names, all by using the
>> loader API in very easy ways.  I don't think these will be common
>> operations, and this seems like a reasonable tradeoff.
>
> I think that defining a module without wanting to register it is the
> common case, at least in real code. Explicit registration should only
> be needed for more ad-hoc configuration scenarios, or generated by
> tools creating bundles. In all other cases, what you refer to with
> external names are actual external resources (including logical
> package names).

Needing to explicitly register a module will certainly be common in a
node-like system, where small modules are the norm. Which is the
common case in general isn't a question we can answer today.  But I
don't think that needing to *avoid* registering will be common.

>>> * Language-level naming semantics interferes with file system/URL
>>> naming semantics.
>>
>> While it's sad that everyone doesn't use the same filesystem ;), this
>> is inevitable unless we force everyone to write *both* internal and
>> external names explicitly for every module; that's clearly more
>> user-hostile than the alternative.
>
> But you do not want to do that for every module! In fact, you rarely
> need to explicitly define a module with an external name at all, at
> least as I envision it. You only want to do that for a module that you
> want to "export" from your current script, so to speak. Such "export"
> (i.e., globally registering a module you defined textually) should be
> a very rare operation to write manually (as mentioned, you don't
> usually do the equivalent in AMD, for example).

Anonymous module defintions are often used in AMD for modules that are
*implicitly* named by the file they're in.  In ES6, that simply
doesn't require a wrapper.

>>> * Bundling ("concatenation") generally would require embedding
>>> arbitrary string resources, not syntactic modules.
>>
>> The reason to use concatenation is to avoid consuming excessive client
>> resources -- in this setting of course you won't want to run
>> translation on the client side.  Translation hooks are important (a)
>> in less perf-sensitive settings like development and (b) for isolating
>> remotely-loaded code, neither of which require concatenation.
>
> I don't believe it's as clear cut, and I can imagine a number of cool
> use cases for translation that do not necessarily fall into this
> simple pattern.

Can you say more about these use cases?

>>> * Module "declarations" are not a declarative mechanism, but an operational one.
>>
>> This comes back to my original point.  Registration of module names is
>> about coordination, and thus this is an import feature, not a problem.
>
> I agree it's an important feature -- but one already provided by the
> loader API. I am not convinced that there is a need to also provide it
> in declarative disguise (without really being declarative) -- and
> certainly not to conflate it with the (actually declarative) notion of
> module definition.

Using the loader API should not be required for declaring a simple
module that other modules import.  This makes life harder for
compilers, tools, and most of all, developers.

Sam


More information about the es-discuss mailing list