Modules and dependencies known before the load

Ian Hickson ian at hixie.ch
Thu Aug 14 14:20:10 PDT 2014


On Wed, 13 Aug 2014, John Barton wrote:
> On Wed, Aug 13, 2014 at 3:52 PM, Ian Hickson <ian at hixie.ch> wrote:
> > On Wed, 13 Aug 2014, John Barton wrote:
> > > On Wed, Aug 13, 2014 at 2:59 PM, Ian Hickson <ian at hixie.ch> wrote:
> > > >
> > > > Suppose a page has this markup:
> > > >
> > > >    <script type=module id=jquery href="jquery.js" whenneeded></script>
> > > >    <script type=module id=a href="a.js" uses="jquery" whenneeded></script>
> > > >    <script type=module>
> > > >     import "a";
> > > >     // ...
> > > >    </script>
> > >
> > > Your example will just work if you just use
> > >
> > >    <script type=module>
> > >     import "a";
> > >     // ...
> > >    </script>
> > >
> > > When this module is compiled the other two will be loaded and 
> > > compiled.
> >
> > That assumes that you have some sort of server-side support so that 
> > the server knows that when you request "a", it should send "a.js" and 
> > "jquery.js", which is an assumption that will work for some people but 
> > not [...]
>
> No, the loader just requests each of these when it needs them.

Right. What I'm trying to do is have the loader load them _before_ it 
needs them.

Without server-side support, on-demand loading looks like this:

      Client   Server
        |        |
  0ms   |-_      | Client finds it needs a.js
        |  -_    |
        |    -_  |
        |      -_| Server sends back a.js
        |     _- |
        |   _-   |
        | _-     |
200ms   |-_      | Client finds it needs jquery.js
        |  -_    |
        |    -_  |
        |      -_| Server sends back jquery.js
        |     _- |
        |   _-   |
        | _-     |
400ms   |-       | Client runs the scripts

...whereas if the client is aware of the dependencies early, it can do:

      Client   Server
        |        |
  0ms   |-_      | Client finds it needs a.js and jquery.js
        |  -_    |
        |    -_  |
        |      -_| Server sends back a.js and jquery.js
        |     _- |
        |   _-   |
        | _-     |
200ms   |-       | Client runs the scripts

...which is a lot faster.


> > everyone. One of the big pieces of feedback I am dealing with on the 
> > HTML side is a request for a way to predeclare the dependency graph to 
> > avoid this problem (so that the browser knows to request a.js and 
> > jquery.js at
> 
> Actually I guess you mean "avoid multi-pass requests for the entire 
> dependency graph".

I'm not sure what that means. But if that means the same thing, then sure. 
The key is just that the client can request all the files it's going to 
need at one time, so that, even without server support, the client doesn't 
wait more than one total RTT.


> > (If you don't have either server-side support or client-side 
> > dependency predeclaration, you end up having to discover the 
> > dependency chart at runtime, which is a huge latency loss.)
> 
> Well, as they said back when I worked on a dairy, "you've step in it".
> 
> This issue is at the root of the split between commonjs (modules for 
> node, synchronous loading) and requirejs (modules for browser, prebuilt 
> into bundles to avoid latency). The ES module system attempts to solve 
> this by providing an asynchronous API (eg System.import()) and a 
> proposal, to support server side dependency bundling in HTTP++.
> 
> The workaround we have now is to pre-build bundles and use the locate() 
> function to map module ids into bundles.  That is basically the 
> requirejs solution (with a much better programmer API).  I predict node 
> will extend their System to include a synchronous API. Then the circle 
> will be complete.
> 
> To be sure this is a difficult or even fundamental problem. I respect 
> all efforts to solve this problem; don't believe anyone who says the 
> solution is simple.

Relying on packages doesn't really scale, unfortunately. When you have 
thousands of modules, as e.g. Google+ does, some of which are script and 
some are style sheets and some are sprite sheets and some are Web 
components and so on, you really need to be able to tell the server "we 
need these files", and you need to be able to do this without having to 
discover the dependencies once you've fetched the first batch of files.


On Thu, 14 Aug 2014, Juan Ignacio Dopazo wrote:
>
> For prefetching we're calling LoadModule() for all the dependencies we 
> want to prefetch in the Fetch or Locate hooks. For example, Systemjs has 
> a System.depCache option which allows you to define dependencies ahead 
> of time. Here's the hook 
> implementation: https://github.com/systemjs/systemjs/blob/master/lib/extension-depCache.js. 

Interesting, ok. So basically you have to manage the dependency tree 
outside of the ES6 Loader mechanism proper? I'm really hoping we can find 
a way to avoid the redundancy of having two mechanisms managing 
dependencies.


On Wed, 13 Aug 2014, Kevin Smith wrote:
> 
> I see two related abilities here, and I'm not quite sure if you want one 
> of them or both:
> 
> 1) The ability to add nodes (and edges) to the dependency graph as the 
> graph is traversed by load operations.
> 
> For example, the user might declaratively indicate that a module depends 
> upon some stylesheet.  You might want to add a node to the graph 
> representing that stylesheet.

That would be good, yes.


> The structure of the graph has an effect on order-of-execution, though, 
> so adding edges can result in observable changes.

Can you elaborate on this, just so I'm sure I understand it right?


> 2) The ability to prefetch resources, given a URL which is currently
> loading.
> 
> For example, the user might want to specify up-front a portion of the
> dependency graph so that resources in that graph can be downloaded in
> parallel.

Isn't that just something you can do one you have (1)? I'm not sure I'm 
following the distinction.


> I don't think the current Loader spec allows either of those abilities. 

Would it make sense to add the first, at least?


> And it might be that the second doesn't really belong in the loader 
> proper.

Yeah, the second would be in HTML.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'


More information about the es-discuss mailing list