how many async-modules can js-app practically load?

kai zhu kaizhu256 at gmail.com
Sun Jun 2 03:53:54 UTC 2019


i apologize for poor framing of my questions.  they are still formative,
but i can clarify abit as follows:

1) original-question - is native es-module's async-behavior desirable?
async side-effects are difficult to manage -- i conjecture that
async-loading 20 es-modules (with dependent side-effects) is not practical
for most mortals to handle.
@frederick describes the mechanism for how to hint the brower to pre-fetch
20 es-modules. but if you pre-fetch, then is loading-behavior effectively
synchronous?
@isiah says he has experience loading 50-100 modules, but was unclear
whether they were individual [async] ```<script type="module">``` tags, or
some es5-transpiled rollup.

i may be wrong about everything, as i'm a bit ignorant on what async
actually means in es-modules (and appreciate it, if someone can clarify
that).



2) the second-question about es-module rollups (which you and i are
debating) stemmed from @isiah's response -- if he and everyone-else use
es5-transpiled rollups (which i suspect), then shouldn't it be desirable
for es-modules to natively support rollups as well?  currently, there's no
way to natively rollup multiple es-modules into a single bundle.

this 2nd question also has implications about es-module's async-behavior
(because rollups "load" modules in sync/blocking fashion).  this could
change side-effect behaviors between development-mode (20 [async] ```<script
type="module">``` tags) and production-mode (1 rollup-bundle).  again, i
may be wrong about that, as i'm ignorant about what async actually is in
es-modules.

-kai

On Sat, Jun 1, 2019 at 9:22 PM guest271314 <guest271314 at gmail.com> wrote:

> > it doesn't actually ```import``` 1000+ es-modules inside the
> rollup-file. it just creates one es-module that exports a dictionary -- and
> assigns the dictionary 1000+ vanilla json-objects and functions.
>
> The code provides a means to fetch N resources and export those resources
> within a single object.
>
> > currently, as i'm aware, nobody uses native es-modules in production,
> because it cannot be rolled-up.
> > in practice es-modules are [babel] transpiled down to es5-amd (or
> similar) for rollup-purposes.
> >
> > if we're actually committed to native es-modules, then we either
> > 1) need to depend on embedders like loading-dev at chromium.org to create
> sophisticated cache-systems, or
> > 2) introduce new language-syntax to delimit es-modules for
> rollup-purposes, e.g.
>
> You still have not clearly defined what you mean by "rolled-up". That
> language appears to be a random nickname, not any immutable principle that
> individuals are bound to recognize or observe (even if "rolled-up" were
> some form of a coding style or standard).
>
> Nor is it clear what you mean by "production".
>
> There is no external central committee that stamps code as "production".
> Even if there were no individual is obliged to submit to such a procedure
> nor have any concern for such an arbitrary and irrelevant presumptive
> review of code.
>
> The only observable points are input and output. In general, how output is
> achieved is immaterial. If there are specific restrictions as to how the
> output can be achieved then those restrictions need to be clearly defined.
>
>
>
> The original post asked "how many async-modules can js-app practically
> load?" and mentioned "circular-references" (the thread appears to mainly
> be about one or more coding styles, not code itself) though as yet no code
> has been posted which demonstrates "circular-references" or any other
> coding problem.
>
> On Sun, Jun 2, 2019 at 1:30 AM kai zhu <kaizhu256 at gmail.com> wrote:
>
>> i played around with your code in jsfiddle [1], and understand it a
>> little more.
>> it doesn't actually ```import``` 1000+ es-modules inside the rollup-file.
>> it just creates one es-module that exports a dictionary
>>  -- and assigns the dictionary 1000+ vanilla json-objects and functions.
>>
>> ```js
>> // the "rollup-file" is a single es-module
>> // that exports 1000+ vanilla dictionary-entries
>> const modules = {};
>>
>> // this is not a es-module, nor is it rolled-up (external fetch)
>> modules.image = <await fetch json from gist.github.com>
>>
>> // this is not a [rolled-up] es-module
>> modules.fn = function () {...}
>>
>> // these are not [rolled-up] es-modules
>> Object.assign(modules, <1000 json-entries>)
>>
>> export {modules}
>> ```
>>
>> currently, as i'm aware, nobody uses native es-modules in production,
>> because it cannot be rolled-up.
>> in practice es-modules are [babel] transpiled down to es5-amd (or
>> similar) for rollup-purposes.
>>
>> if we're actually committed to native es-modules, then we either
>> 1) need to depend on embedders like loading-dev at chromium.org to create
>> sophisticated cache-systems, or
>> 2) introduce new language-syntax to delimit es-modules for
>> rollup-purposes, e.g.
>>
>> ```js
>> // rollup.js with [hypothetical] # delimited es-modules
>> # module aa
>> import {bb} as bb;
>> export ...;
>>
>> # module bb
>> export ...;
>> ```
>>
>> i'm generally skeptical of option 1, given how poorly npmjs.com has
>> handled similar problems deduplicating children in node_modules/ directory.
>>
>> [1] jsfiddle pseudo-module rollup
>> https://jsfiddle.net/06twrLfd/
>>
>> On Sat, Jun 1, 2019 at 5:30 PM guest271314 <guest271314 at gmail.com> wrote:
>>
>>> > your rollup solution is interesting,
>>>
>>> What  is "rollup" referring to?
>>>
>>> > but i get an error when run in chrome (i changed to n=20 to prevent
>>> name-collision, but it still happens).
>>>
>>> The duplicate ("collision") entry an ```try..catch``` block is included
>>> in the code to demonstrate given an array of module names to be exported
>>> and imported as identifiers 1) duplicate entries can be filtered; 2) if a
>>> plain object is exported duplicate identifiers ("collision") is not
>>> possible as a JavaScript plain object does not have duplicate property
>>> names ("collision"); if there is an issue with identifiers in a module the
>>> cause would not be the number of async-modules loaded ("how many"), but the
>>> naming of the identifiers within the code, using or not using ```const```
>>> or ```let```. Still not sure what the actual issue is?
>>>
>>> > don't completely understand how it works,
>>>
>>> Use an ```async``` function to fetch data, check for the described
>>> "collision" , create a ```data URI``` to be imported, optionally, append
>>> addition code to be executed within the ```<script type="module">```.
>>>
>>> > but not sure of suitability for production-use, because of its dynamic
>>> <script> tag generation.
>>>
>>> What is the issue with dynamic ```<script>``` tag generation?
>>>
>>> There is more than one possible approach to achieve the presumptive
>>> requirement, that is still not clear to the exclusion of what is not the
>>> expected result.
>>>
>>> There were no restrictions described at the OP and following messages
>>> other than other than
>>>
>>> > pure-es6 application with 20 es-modules rolled-up into one
>>> [production] bundle?
>>>
>>> The example code uses only JavaScript implementation shipped with the
>>> browser without any external, third-party libraries.
>>>
>>> What standard or definition are you relying for the meaning of the term
>>> "production-use"? What procedure are you using to determine if code is
>>> "production-use" "suitable"? How is that procedure related to "how many
>>> async-modules can js-app practically load?"?
>>>
>>>
>>>
>>> On Sat, Jun 1, 2019 at 9:42 PM kai zhu <kaizhu256 at gmail.com> wrote:
>>>
>>>> your rollup solution is interesting, but i get an error when run in
>>>> chrome (i changed to n=20 to prevent name-collision, but it still
>>>> happens).  don't completely understand how it works, but not sure of
>>>> suitability for production-use, because of its dynamic <script> tag
>>>> generation.
>>>>
>>>> ```console
>>>> ReferenceError: module names ["yeqjqb02mvg3yze26rc5"] are not unique
>>>>     at
>>>> data:application/javascript,%0A%20%20%20%20%20%20const%20modules...
>>>> ```
>>>>
>>>> On Sat, Jun 1, 2019 at 2:33 PM guest271314 <guest271314 at gmail.com>
>>>> wrote:
>>>>
>>>>> Re: how many async-modules can js-app practically load?
>>>>>
>>>>> An example of exporting and importing loading 1000 properties in a
>>>>> single module, where duplicate property names are checked for. Since
>>>>> JavaScript plain objects cannot have duplicate property names there should
>>>>> not be any "collisions"; the code can check for and modify the object to be
>>>>> exported, though the last duplicate property name will be exported without
>>>>> any errors thrown unless the code is composed to throw such an error.
>>>>>
>>>>> ```
>>>>>   (async() => {
>>>>>     const oneThousandModules = encodeURIComponent(
>>>>>       // substitute rand for a Set of module names to be exported
>>>>>       // e.g. const moduleNames = ['moduleA', 'moduleB', ...moduleZ]
>>>>>       `
>>>>>       const modules = {};
>>>>>       // set a function to be exported
>>>>>       modules.fn = function() {return 'a function'};
>>>>>       // function to set (1000) 'random' module names to be exported
>>>>>       const rand = (seed = 'abcdefghijklmnopqrstuvwxyz0123456789', n =
>>>>> 5, len = seed.length) =>
>>>>>         '.'.repeat(n).replace(/./g, _ => seed[~~(Math.random() *
>>>>> len)]);
>>>>>         // use Set for unique module identifiers
>>>>>         const moduleNames = [...Array(1000)].map(_ => rand());
>>>>>         const moduleIdentifiers = new Set(moduleNames);
>>>>>         // below line will cause ReferenceError to be thrown
>>>>>         moduleNames.push(moduleNames[0]);
>>>>>         try {
>>>>>           if (moduleIdentifiers.size !== moduleNames.length) {
>>>>>             // check for duplicates
>>>>>             const duplicates = moduleNames.filter((moduleName, index)
>>>>> => moduleNames.indexOf(moduleName) !== index);
>>>>>             // notification of duplicate module names
>>>>>             throw new ReferenceError('module names ' +
>>>>> JSON.stringify(duplicates) + ' are not unique');
>>>>>             // perform the designated task if duplicate module names
>>>>> are found here
>>>>>           }
>>>>>         } catch (e) {
>>>>>           console.error(e);
>>>>>           console.trace();
>>>>>         }
>>>>>         // get, set (sync or async) exported module here
>>>>>         Object.assign(modules, ...[...moduleIdentifiers].map((id,
>>>>> value) => ({[id]:value})));
>>>>>         // since JavaScript plain object cannot have duplicate
>>>>> property names
>>>>>         // modules object will still be exported without duplicate
>>>>> property names
>>>>>         // without collisions
>>>>>         export {modules}
>>>>>     `);
>>>>>     const scriptText = `import {modules} from
>>>>> "data:application/javascript,${oneThousandModules};${encodeURIComponent('console.log(modules);for
>>>>> (const key in modules) {if (typeof modules[key] === \'function\')
>>>>> {console.log(modules[key]());}}')}"`;
>>>>>     const script = document.createElement("script");
>>>>>     script.type = "module";
>>>>>     script.textContent = scriptText;
>>>>>     document.head.appendChild(script);
>>>>>   })();
>>>>> ```
>>>>>
>>>>> plnkr https://plnkr.co/edit/CgEhBY?p=preview
>>>>>
>>>>> On Sat, Jun 1, 2019 at 1:51 AM kai zhu <kaizhu256 at gmail.com> wrote:
>>>>>
>>>>>> > Place all of the code to be exported in 1 file?
>>>>>>
>>>>>> that obviously will not work, because of module-scope collision.  can
>>>>>> anyone share their experience on deploying a [babel-free] pure-es6
>>>>>> application with 20 es-modules rolled-up into one [production] bundle?  is
>>>>>> it even possible?
>>>>>>
>>>>>>
>>>>>> On Fri, May 31, 2019 at 7:55 PM guest271314 <guest271314 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> > how would i transition from development-mode (20 es-module files)
>>>>>>> -> production-mode (1 rollup file)?
>>>>>>>
>>>>>>> Place all of the code to be exported in 1 file?
>>>>>>>
>>>>>>> > with some of them having circular-references
>>>>>>>
>>>>>>> Not certain how that is possible when using ```import``` within
>>>>>>> ```<script type="module">```?
>>>>>>>
>>>>>>> > how many async-modules can js-app practically load?
>>>>>>>
>>>>>>> Again, how many have you tried to load? 100? 500? 1000? Either
>>>>>>> should be possible.
>>>>>>>
>>>>>>> What specific issue are you actually to resolve?
>>>>>>>
>>>>>>> On Fri, May 31, 2019 at 5:40 PM kai zhu <kaizhu256 at gmail.com> wrote:
>>>>>>>
>>>>>>>> > Oh, and yes, I've loaded upwards of 50-100 modules in
>>>>>>>> development. 20
>>>>>>>> modules is *easy* to achieve in single-page apps.
>>>>>>>>
>>>>>>>> was that with some combination of babel/rollup/webpack or pure-es6?
>>>>>>>> if i want to develop a pure-es6 webapp (no babel), how would i
>>>>>>>> transition from development-mode (20 es-module files) -> production-mode (1
>>>>>>>> rollup file)?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, May 31, 2019 at 10:47 AM Isiah Meadows <
>>>>>>>> isiahmeadows at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> If it's bundled by Rollup or Webpack into a single bundle, it's
>>>>>>>>> equivalent to a single `<script type="module" src="...">` pointing
>>>>>>>>> towards the original entry point, excluding network requests.* But
>>>>>>>>> in
>>>>>>>>> either case, you aren't listing 50 scripts, you're only listing the
>>>>>>>>> entry module and importing child modules within parent modules.
>>>>>>>>> Rollup
>>>>>>>>> and Webpack do mostly the same thing browsers do when it comes to
>>>>>>>>> resolving dependencies, just they generate a bundle afterwards
>>>>>>>>> where
>>>>>>>>> browsers execute code afterwards. Also, it's worth noting that the
>>>>>>>>> gap
>>>>>>>>> between a single large request and multiple smaller requests has
>>>>>>>>> shrunk a lot since HTTP/2 came along, since it's binary, it allows
>>>>>>>>> requests and response data to be interleaved, it better leverages
>>>>>>>>> the
>>>>>>>>> underlying TCP protocol format, and it allows servers to send data
>>>>>>>>> pre-emptively without the client requesting it first. (Web sockets
>>>>>>>>> are
>>>>>>>>> built on this functionality.) It's still better to bundle in
>>>>>>>>> general,
>>>>>>>>> but it's less of a problem not to.
>>>>>>>>>
>>>>>>>>> This is *not* the case for `<script type="module">` elements -
>>>>>>>>> those
>>>>>>>>> operate more like inline scripts that happen to have the ability to
>>>>>>>>> `import`.
>>>>>>>>>
>>>>>>>>> Oh, and yes, I've loaded upwards of 50-100 modules in development.
>>>>>>>>> 20
>>>>>>>>> modules is *easy* to achieve in single-page apps.
>>>>>>>>>
>>>>>>>>> * This is, of course, not the case if you are using pure ES6 and
>>>>>>>>> you
>>>>>>>>> aren't using any plugins to, say, run the original source through
>>>>>>>>> Babel for React + JSX or something.
>>>>>>>>>
>>>>>>>>> -----
>>>>>>>>>
>>>>>>>>> Isiah Meadows
>>>>>>>>> contact at isiahmeadows.com
>>>>>>>>> www.isiahmeadows.com
>>>>>>>>> On Sat, May 25, 2019 at 2:12 AM kai zhu <kaizhu256 at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > Asynchronous loading differs only in
>>>>>>>>> > that it takes more code to express the same logic and you have
>>>>>>>>> to take
>>>>>>>>> > into account concurrent requests (and you need to cache the
>>>>>>>>> request,
>>>>>>>>> > not the result), but it's otherwise the same from 1km away.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > so async-loading 50 ```<script type="module">``` tags
>>>>>>>>> > has equivalent side-effect
>>>>>>>>> > as sync-loading single webpack-rollup (of same 50 modules)?
>>>>>>>>> >
>>>>>>>>> > i have nagging suspicion of doubts.  has anyone tried native
>>>>>>>>> async-loading large numbers (>10) of
>>>>>>>>> > ```<script type="module">``` tags, and verify it resolves
>>>>>>>>> identically to using a single webpack-rollup?
>>>>>>>>> >
>>>>>>>>> > again, i'm not that knowledgeable on es-modules, so above
>>>>>>>>> question may be trivially true, and i'm just not aware.
>>>>>>>>> >
>>>>>>>>> > -kai
>>>>>>>>> >
>>>>>>>>> > On 24 May 2019, at 23:41, Isiah Meadows <isiahmeadows at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > There's two main reasons why it scales:
>>>>>>>>> >
>>>>>>>>> > 1. Modules are strongly encapsulated while minimizing global
>>>>>>>>> pollution.
>>>>>>>>> > 2. The resolution algorithm applies the same logic no matter how
>>>>>>>>> many
>>>>>>>>> > modules are loaded.
>>>>>>>>> >
>>>>>>>>> > It's much easier for it to scale when you write the code unaware
>>>>>>>>> of
>>>>>>>>> > how many modules you might be loading and unaware of how deep
>>>>>>>>> their
>>>>>>>>> > dependency graph is. Fewer assumptions here is key. It's an
>>>>>>>>> > engineering problem, but a relatively simple one.
>>>>>>>>> >
>>>>>>>>> > If you want a short example of how sync module resolution works,
>>>>>>>>> you
>>>>>>>>> > can take a look at this little utility I wrote:
>>>>>>>>> > https://github.com/isiahmeadows/simple-require-loader. That
>>>>>>>>> doesn't
>>>>>>>>> > asynchronously resolve modules, but it should help explain the
>>>>>>>>> process
>>>>>>>>> > from a synchronous standpoint. Asynchronous loading differs only
>>>>>>>>> in
>>>>>>>>> > that it takes more code to express the same logic and you have
>>>>>>>>> to take
>>>>>>>>> > into account concurrent requests (and you need to cache the
>>>>>>>>> request,
>>>>>>>>> > not the result), but it's otherwise the same from 1km away.
>>>>>>>>> >
>>>>>>>>> > -----
>>>>>>>>> >
>>>>>>>>> > Isiah Meadows
>>>>>>>>> > contact at isiahmeadows.com
>>>>>>>>> > www.isiahmeadows.com
>>>>>>>>> >
>>>>>>>>> > On Thu, May 23, 2019 at 10:49 AM kai zhu <kaizhu256 at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > actually, i admit i don't know what i'm talking about.  just
>>>>>>>>> generally confused (through ignorance) on how large-scale es-module
>>>>>>>>> dependencies resolve when loaded/imported asynchronously.
>>>>>>>>> >
>>>>>>>>> > On Wed, May 22, 2019 at 10:42 PM Logan Smyth <
>>>>>>>>> loganfsmyth at gmail.com> wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > Can you elaborate on what loading state you need to keep track
>>>>>>>>> of? What is the bottleneck that you run into? Also to be sure, when you say
>>>>>>>>> async-load, do you mean `import()`?
>>>>>>>>> >
>>>>>>>>> > On Wed, May 22, 2019, 20:17 kai zhu <kaizhu256 at gmail.com> wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > i don't use es-modules.
>>>>>>>>> > but with amd/requirejs, I start having trouble with
>>>>>>>>> module-initializations in nodejs/browser at ~5 async modules (that may or
>>>>>>>>> may not have circular-references).  10 would be hard, and 20 would be near
>>>>>>>>> inhuman for me.
>>>>>>>>> >
>>>>>>>>> > can we say its somewhat impractical for most applications to
>>>>>>>>> load more than 50 async modules (with some of them having
>>>>>>>>> circular-references)?  and perhaps better design/spec module-loading
>>>>>>>>> mechanisms with this usability concern in mind?
>>>>>>>>> >
>>>>>>>>> > p.s. its also impractical for me to async-load 5 or more modules
>>>>>>>>> without using globalThis to keep track of each module's loading-state.
>>>>>>>>> > _______________________________________________
>>>>>>>>> > es-discuss mailing list
>>>>>>>>> > es-discuss at mozilla.org
>>>>>>>>> > https://mail.mozilla.org/listinfo/es-discuss
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > _______________________________________________
>>>>>>>>> > es-discuss mailing list
>>>>>>>>> > es-discuss at mozilla.org
>>>>>>>>> > https://mail.mozilla.org/listinfo/es-discuss
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> es-discuss mailing list
>>>>>>>> es-discuss at mozilla.org
>>>>>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>>>>>
>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20190601/4da73810/attachment-0001.html>


More information about the es-discuss mailing list