Module naming and declarations
rossberg at google.com
Fri May 10 07:18:56 PDT 2013
On 9 May 2013 08:47, David Herman <dherman at mozilla.com> wrote:
> On May 8, 2013, at 8:05 AM, Andreas Rossberg <rossberg at google.com> wrote:
>> You seem to believe otherwise, but I think you still need to explain
>> how any of the above cases is not sufficiently (or even superiorly)
>> supported by lexical modules + the loader API.
> The most important flaw of this is staging. The loader API lets you *dynamically* modify the registry, but those changes cannot be used by code compiled at the same time as the code that does the modification.
I know. It just seems that it isn't actually a problem, because all
interesting use cases require staging anyway. For example, you cannot
use loader.ondemand without staging, and that's what you'll have to
use for bundles. If loader.ondemand has the semantics I expected it to
have (see below), that is all the staging you need to do manually.
>> In any case, I don't see how this observation necessarily implies
>> _anything_ for the form of module *declarations*. I suspect that your
>> thinking had very much to do with (naive) concatenation, but
>> concatenation without a tool will never be possible for modules
>> anyway. So it really is a totally moot point.
> I think it's important not just to be able to write bundlers, but to be able to do the transformation in a fine-grained way. It should be possible to move module declarations around by hand, it should be possible for servers to make on-the-fly decisions about where to place module declarations, etc. And it should be possible to do this without greatly perturbing the shape of the resulting code.
Can you explain how one form of module declaration is easier to "move
around"? In a single script there surely is no difference.
If you are talking about moving modules across scripts/files, or even
turning files into module declarations, then I don't see how that can
possibly 'just work' in general, even in your approach. It seems you
will need some support from your bundling/optimization/configuration
tool, and/or you will need to adjust the set-up of your loader
appropriately (i.e., set up the appropriate .ondemand entries). And
once you have to go there, how does it make a difference?
> Put differently, controlling where and when code is bundled together is something that sophisticated web applications do often and sometimes at fine granularity, and it's done by experts at web development who shouldn't have to be experts at writing compilers or reading their output. Consider Eric Ferraiuolo's examples of servers making decisions to speculatively bundle additional modules in a response to a request. These are decisions that are about network efficiency, and they shouldn't have to deal with code transformations at the same time.
Again, I don't see how any of that would be affected. Can you be more
specific about what you think would not work? Even when done
dynamically, the framework will need to know how to create bundles,
and will have to implement a certain amount of rewriting. Moreover,
nothing prevents you from creating a fully dynamic bundle, even in the
approach I suggest -- it just doesn't force you to (and in most cases,
you probably don't want it.)
On a more general note, I think you are trying to solve problems on
the language level here for which that simply is the wrong attack
mentioned before, real applications like Gmail, which very definitely
need to deal with these problems, won't be helped by a solution that
can handle only JS resources.
>> OK, let's get concrete and see how this works. Assume you are writing
>> some subsystem or library. There are three phases.
>> 1. Development. Most modules will typically be in individual files,
>> that you import through external names...
>> 2. Deployment. Once you're ready to release, you want to bundle
>> everything into one file ("concatenation"). Either way, that requires
>> a tool.
> You don't necessarily bundle *everything* into one file. Large-scale apps may bundle things in various ways.
Sure. For that reason, I was specifically talking about a library or
subsystem. A bundle can still contain external references to modules
outside that bundle.
>> With lexical module declarations, the tool will be similar, but
>> distinguishes the set of module paths that you intend to export from
>> the package. The resulting file will again contain all individual
>> files wrapped into module declarations, but this time with lexical
>> names (generated by some means -- the details don't matter since there
>> is no intention of making them accessible; you could even wrap
>> everything into an anonymous module). All import file names referring
>> to non-exported modules are replaced by their lexical references. In
>> addition, the tool generates a couple of suitable calls to loader.set
>> for registering the exported modules. The names for those are picked
>> in the same manner as in the other approach.
> As I said above, this is broken. If we don't provide a declarative way to register modules, then they have to be added to the registry *in a different stage* from any code that uses them. This forces you to sequentialize the loading of different packages of your application, which is a non-starter.
I'm still not sure what your argument is. Yes, you have to use
staging. But no, you cannot avoid it, declarative registration or not.
In any use case involving partial bundling, you will always need
staging. You have to use either separate script tags, which are staged
by definition, or invoke loader.ondemand in a prior stage.
(Except in the edge case where you actually bundle everything in one
file. In that case, obviously, you don't need staging either way.)
> What's more, when you start from your assumption #1 (and I do) that most modules will be in separate files, then lexical modules aren't buying the user much at all. Except for the cases where it makes sense to have small sub-modules that fit within a single file. As I've said before, I can imagine this being occasionally useful, but it's simply not important enough for ES6.
I think I've explained the reasons at great lengths already at the
beginning of this thread, and at various points in the middle. I'm
happy to reiterate, but these posts are already getting long...
>> The advantage of the latter approach is again that all intra-bundle
>> references can be (1) cheap and reliable
> Sounds like you're misinterpreting the semantics of static linking. I have clarified this several times already. References between ES6 modules are not dynamically indirected through a dynamic table. They are statically resolved at link time. Linked modules are every bit as cheap and reliable as lexical references.
I don't think I'm misinterpreting. You are talking about linked
modules. The problem is, in your approach, all modules start out as
unlinked, and I can only link them through unreliable dynamic
references. As I've pointed out in my OP, there is no way in which
your proposal allows me to choose that the thing I link is the one
I've actually defined on the previous line. That is bad for all the
well-known reasons, and the global object story should make us wary.
(So far, the only counter argument I have heard was "but a loader can
mess with your source code anyway". But that's reductio ad absurdum,
and could be used to argue against sane language semantics for
>> and (2) don't pollute the
>> global namespace. At the same time, you are free to export as many of
>> the modules as you like. So this approach is strictly more powerful,
>> the user has control.
> Except that as I explained above, without a way to register a module name declaratively, this forces you to load dependencies sequentially, which is unacceptable.
See above and below.
> It also couples structural containment ("module X is conceptually part of package Y") with textual containment, which is strictly less flexible for controlling the delivery of the source over the wire.
Why should that cause any such coupling? You'd still be able to do the
same sort of exports as before. You just don't _have to_.
>> 3. Usage. In both approaches, the bundled file you have created
>> contains a number of modules with external (logical) names. There are
>> two ways to fetch them in another program: (a) either through a script
>> tag, or (b) by calling loader.ondemand with the list of provided
>> names. Both ways work equally for either of the designs!
> Doing it through a script tag is sequentializing. I don't understand how loader.ondemand works if the contents of the file has to do a dynamic loader.set. Once again, this sounds broken from a staging perspective.
OK, perhaps I am under wrong assumptions about the semantics of
ondemand. I assumed that when you first import a module that has been
registered via ondemand, the respective script will be executed,
recursively, and then linking of the outer script continues, taking
the updated loader environment into account.
If that assumption is true, then all the staging you need for my
suggestion to work is already inherent in ondemand.
If my assumption is not true, then I have a couple of questions:
(1) What _is_ the semantics of ondemand? When is the script body executed?
(2) Why is that semantics needed? Or IOW, what would be the practical
problem with the semantics I assumed?
(3) Would you agree that, with the semantics I assumed, my suggestion
(4) Wouldn't the semantics I assumed be more consistent with the model
that we have been discussing for imports from AMD-like module systems
at the end of the last meeting?
I can think of one possible answer to (2), and that's cross-package
mutual recursion. But I believe you agreed in March that that does not
seem to be common practice, and there's no need to support it.
More information about the es-discuss