Modules and dependencies known before the load

John Barton johnjbarton at google.com
Thu Aug 14 14:46:24 PDT 2014


On Thu, Aug 14, 2014 at 2:20 PM, Ian Hickson <ian at hixie.ch> wrote:

> On Wed, 13 Aug 2014, John Barton wrote:
> > On Wed, Aug 13, 2014 at 3:52 PM, Ian Hickson <ian at hixie.ch> wrote:
> > > On Wed, 13 Aug 2014, John Barton wrote:
> > > > On Wed, Aug 13, 2014 at 2:59 PM, Ian Hickson <ian at hixie.ch> wrote:
> > > > >
> > > > > Suppose a page has this markup:
> > > > >
> > > > >    <script type=module id=jquery href="jquery.js"
> whenneeded></script>
> > > > >    <script type=module id=a href="a.js" uses="jquery"
> whenneeded></script>
> > > > >    <script type=module>
> > > > >     import "a";
> > > > >     // ...
> > > > >    </script>
> > > >
> > > > Your example will just work if you just use
> > > >
> > > >    <script type=module>
> > > >     import "a";
> > > >     // ...
> > > >    </script>
> > > >
> > > > When this module is compiled the other two will be loaded and
> > > > compiled.
> > >
> > > That assumes that you have some sort of server-side support so that
> > > the server knows that when you request "a", it should send "a.js" and
> > > "jquery.js", which is an assumption that will work for some people but
> > > not [...]
> >
> > No, the loader just requests each of these when it needs them.
>
> Right. What I'm trying to do is have the loader load them _before_ it
> needs them.
>
> Without server-side support, on-demand loading looks like this:
>
>       Client   Server
>         |        |
>   0ms   |-_      | Client finds it needs a.js
>         |  -_    |
>         |    -_  |
>         |      -_| Server sends back a.js
>         |     _- |
>         |   _-   |
>         | _-     |
> 200ms   |-_      | Client finds it needs jquery.js
>         |  -_    |
>         |    -_  |
>         |      -_| Server sends back jquery.js
>         |     _- |
>         |   _-   |
>         | _-     |
> 400ms   |-       | Client runs the scripts
>
> ...whereas if the client is aware of the dependencies early, it can do:
>
>       Client   Server
>         |        |
>   0ms   |-_      | Client finds it needs a.js and jquery.js
>         |  -_    |
>         |    -_  |
>         |      -_| Server sends back a.js and jquery.js
>         |     _- |
>         |   _-   |
>         | _-     |
> 200ms   |-       | Client runs the scripts
>
> ...which is a lot faster.
>

But since the only way the client can know that it needs a.js and jquery.js
is if the server tells it, then:

      Client   Server sends  a.js and jquery.js
        |        |
  0ms   |-_      | Client finds it has a.js and jquery.js
 0ms   |-       | Client runs the scripts

....which is even faster.


>
> > > everyone. One of the big pieces of feedback I am dealing with on the
> > > HTML side is a request for a way to predeclare the dependency graph to
> > > avoid this problem (so that the browser knows to request a.js and
> > > jquery.js at
> >
> > Actually I guess you mean "avoid multi-pass requests for the entire
> > dependency graph".
>
> I'm not sure what that means. But if that means the same thing, then sure.
>

Yes, your example just happens to be one layer deep and thus requires just
one extra trip.


> The key is just that the client can request all the files it's going to
> need at one time, so that, even without server support, the client doesn't
> wait more than one total RTT.
>

Wait, you claimed earlier that without server support it's extra round
trips. Now without server support it's no extra round trips.  Which one ;-)

If you meant "suppose we had magic file on the server which listed
dependencies and a Loader that read that file, then we can succeed by
building the magic file and don't need server code changes".

That's what "bundling" does: builds a magic file and ships all of the
required modules in dependency order to the server. It even skips the step
where the list it built and just sends the all the dependencies.

Could we have better tools that allowed finer grained control over the
bundle size vs trips? Sure.


>
>
> > > (If you don't have either server-side support or client-side
> > > dependency predeclaration, you end up having to discover the
> > > dependency chart at runtime, which is a huge latency loss.)
> >
> > Well, as they said back when I worked on a dairy, "you've step in it".
> >
> > This issue is at the root of the split between commonjs (modules for
> > node, synchronous loading) and requirejs (modules for browser, prebuilt
> > into bundles to avoid latency). The ES module system attempts to solve
> > this by providing an asynchronous API (eg System.import()) and a
> > proposal, to support server side dependency bundling in HTTP++.
> >
> > The workaround we have now is to pre-build bundles and use the locate()
> > function to map module ids into bundles.  That is basically the
> > requirejs solution (with a much better programmer API).  I predict node
> > will extend their System to include a synchronous API. Then the circle
> > will be complete.
> >
> > To be sure this is a difficult or even fundamental problem. I respect
> > all efforts to solve this problem; don't believe anyone who says the
> > solution is simple.
>
> Relying on packages doesn't really scale, unfortunately. When you have
> thousands of modules, as e.g. Google+ does, some of which are script and
> some are style sheets and some are sprite sheets and some are Web
> components and so on, you really need to be able to tell the server "we
> need these files", and you need to be able to do this without having to
> discover the dependencies once you've fetched the first batch of files.
>

Which is why Google+ uses bundling.

I don't know what you mean by "packages" but I mean a collection of all the
modules required by one root application or module. That is the sense that
npm means packages and it is closely related to the bundling idea. We could
see the emergence of packages for browser or cross-platform; it's likely
that many applications can be developed quickly and have good enough
performance in browser if they relied on bundles==packages. By that I mean
they would use pre-bundled sets of modules or they can re-bundle packages
that are themselves bundles. And some of these packages could be shared
between browser and node devs.

However I find that neither node devs nor browser devs have interest in
this direction.  Node devs want npm modules using require(); browser devs
want no bundles (for development speed) or highly-optimized bundles.  And
es-discuss does not want to discuss bundles/packaging in any case.

jjb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20140814/481e7079/attachment-0001.html>


More information about the es-discuss mailing list