how many async-modules can js-app practically load?

kai zhu kaizhu256 at gmail.com
Sun Jun 2 01:30:43 UTC 2019


i played around with your code in jsfiddle [1], and understand it a little
more.
it doesn't actually ```import``` 1000+ es-modules inside the rollup-file.
it just creates one es-module that exports a dictionary
 -- and assigns the dictionary 1000+ vanilla json-objects and functions.

```js
// the "rollup-file" is a single es-module
// that exports 1000+ vanilla dictionary-entries
const modules = {};

// this is not a es-module, nor is it rolled-up (external fetch)
modules.image = <await fetch json from gist.github.com>

// this is not a [rolled-up] es-module
modules.fn = function () {...}

// these are not [rolled-up] es-modules
Object.assign(modules, <1000 json-entries>)

export {modules}
```

currently, as i'm aware, nobody uses native es-modules in production,
because it cannot be rolled-up.
in practice es-modules are [babel] transpiled down to es5-amd (or similar)
for rollup-purposes.

if we're actually committed to native es-modules, then we either
1) need to depend on embedders like loading-dev at chromium.org to create
sophisticated cache-systems, or
2) introduce new language-syntax to delimit es-modules for rollup-purposes,
e.g.

```js
// rollup.js with [hypothetical] # delimited es-modules
# module aa
import {bb} as bb;
export ...;

# module bb
export ...;
```

i'm generally skeptical of option 1, given how poorly npmjs.com has handled
similar problems deduplicating children in node_modules/ directory.

[1] jsfiddle pseudo-module rollup
https://jsfiddle.net/06twrLfd/

On Sat, Jun 1, 2019 at 5:30 PM guest271314 <guest271314 at gmail.com> wrote:

> > your rollup solution is interesting,
>
> What  is "rollup" referring to?
>
> > but i get an error when run in chrome (i changed to n=20 to prevent
> name-collision, but it still happens).
>
> The duplicate ("collision") entry an ```try..catch``` block is included in
> the code to demonstrate given an array of module names to be exported and
> imported as identifiers 1) duplicate entries can be filtered; 2) if a plain
> object is exported duplicate identifiers ("collision") is not possible as a
> JavaScript plain object does not have duplicate property names
> ("collision"); if there is an issue with identifiers in a module the cause
> would not be the number of async-modules loaded ("how many"), but the
> naming of the identifiers within the code, using or not using ```const```
> or ```let```. Still not sure what the actual issue is?
>
> > don't completely understand how it works,
>
> Use an ```async``` function to fetch data, check for the described
> "collision" , create a ```data URI``` to be imported, optionally, append
> addition code to be executed within the ```<script type="module">```.
>
> > but not sure of suitability for production-use, because of its dynamic
> <script> tag generation.
>
> What is the issue with dynamic ```<script>``` tag generation?
>
> There is more than one possible approach to achieve the presumptive
> requirement, that is still not clear to the exclusion of what is not the
> expected result.
>
> There were no restrictions described at the OP and following messages
> other than other than
>
> > pure-es6 application with 20 es-modules rolled-up into one [production]
> bundle?
>
> The example code uses only JavaScript implementation shipped with the
> browser without any external, third-party libraries.
>
> What standard or definition are you relying for the meaning of the term
> "production-use"? What procedure are you using to determine if code is
> "production-use" "suitable"? How is that procedure related to "how many
> async-modules can js-app practically load?"?
>
>
>
> On Sat, Jun 1, 2019 at 9:42 PM kai zhu <kaizhu256 at gmail.com> wrote:
>
>> your rollup solution is interesting, but i get an error when run in
>> chrome (i changed to n=20 to prevent name-collision, but it still
>> happens).  don't completely understand how it works, but not sure of
>> suitability for production-use, because of its dynamic <script> tag
>> generation.
>>
>> ```console
>> ReferenceError: module names ["yeqjqb02mvg3yze26rc5"] are not unique
>>     at data:application/javascript,%0A%20%20%20%20%20%20const%20modules...
>> ```
>>
>> On Sat, Jun 1, 2019 at 2:33 PM guest271314 <guest271314 at gmail.com> wrote:
>>
>>> Re: how many async-modules can js-app practically load?
>>>
>>> An example of exporting and importing loading 1000 properties in a
>>> single module, where duplicate property names are checked for. Since
>>> JavaScript plain objects cannot have duplicate property names there should
>>> not be any "collisions"; the code can check for and modify the object to be
>>> exported, though the last duplicate property name will be exported without
>>> any errors thrown unless the code is composed to throw such an error.
>>>
>>> ```
>>>   (async() => {
>>>     const oneThousandModules = encodeURIComponent(
>>>       // substitute rand for a Set of module names to be exported
>>>       // e.g. const moduleNames = ['moduleA', 'moduleB', ...moduleZ]
>>>       `
>>>       const modules = {};
>>>       // set a function to be exported
>>>       modules.fn = function() {return 'a function'};
>>>       // function to set (1000) 'random' module names to be exported
>>>       const rand = (seed = 'abcdefghijklmnopqrstuvwxyz0123456789', n =
>>> 5, len = seed.length) =>
>>>         '.'.repeat(n).replace(/./g, _ => seed[~~(Math.random() * len)]);
>>>         // use Set for unique module identifiers
>>>         const moduleNames = [...Array(1000)].map(_ => rand());
>>>         const moduleIdentifiers = new Set(moduleNames);
>>>         // below line will cause ReferenceError to be thrown
>>>         moduleNames.push(moduleNames[0]);
>>>         try {
>>>           if (moduleIdentifiers.size !== moduleNames.length) {
>>>             // check for duplicates
>>>             const duplicates = moduleNames.filter((moduleName, index) =>
>>> moduleNames.indexOf(moduleName) !== index);
>>>             // notification of duplicate module names
>>>             throw new ReferenceError('module names ' +
>>> JSON.stringify(duplicates) + ' are not unique');
>>>             // perform the designated task if duplicate module names are
>>> found here
>>>           }
>>>         } catch (e) {
>>>           console.error(e);
>>>           console.trace();
>>>         }
>>>         // get, set (sync or async) exported module here
>>>         Object.assign(modules, ...[...moduleIdentifiers].map((id, value)
>>> => ({[id]:value})));
>>>         // since JavaScript plain object cannot have duplicate property
>>> names
>>>         // modules object will still be exported without duplicate
>>> property names
>>>         // without collisions
>>>         export {modules}
>>>     `);
>>>     const scriptText = `import {modules} from
>>> "data:application/javascript,${oneThousandModules};${encodeURIComponent('console.log(modules);for
>>> (const key in modules) {if (typeof modules[key] === \'function\')
>>> {console.log(modules[key]());}}')}"`;
>>>     const script = document.createElement("script");
>>>     script.type = "module";
>>>     script.textContent = scriptText;
>>>     document.head.appendChild(script);
>>>   })();
>>> ```
>>>
>>> plnkr https://plnkr.co/edit/CgEhBY?p=preview
>>>
>>> On Sat, Jun 1, 2019 at 1:51 AM kai zhu <kaizhu256 at gmail.com> wrote:
>>>
>>>> > Place all of the code to be exported in 1 file?
>>>>
>>>> that obviously will not work, because of module-scope collision.  can
>>>> anyone share their experience on deploying a [babel-free] pure-es6
>>>> application with 20 es-modules rolled-up into one [production] bundle?  is
>>>> it even possible?
>>>>
>>>>
>>>> On Fri, May 31, 2019 at 7:55 PM guest271314 <guest271314 at gmail.com>
>>>> wrote:
>>>>
>>>>> > how would i transition from development-mode (20 es-module files)
>>>>> -> production-mode (1 rollup file)?
>>>>>
>>>>> Place all of the code to be exported in 1 file?
>>>>>
>>>>> > with some of them having circular-references
>>>>>
>>>>> Not certain how that is possible when using ```import``` within
>>>>> ```<script type="module">```?
>>>>>
>>>>> > how many async-modules can js-app practically load?
>>>>>
>>>>> Again, how many have you tried to load? 100? 500? 1000? Either should
>>>>> be possible.
>>>>>
>>>>> What specific issue are you actually to resolve?
>>>>>
>>>>> On Fri, May 31, 2019 at 5:40 PM kai zhu <kaizhu256 at gmail.com> wrote:
>>>>>
>>>>>> > Oh, and yes, I've loaded upwards of 50-100 modules in development.
>>>>>> 20
>>>>>> modules is *easy* to achieve in single-page apps.
>>>>>>
>>>>>> was that with some combination of babel/rollup/webpack or pure-es6?
>>>>>> if i want to develop a pure-es6 webapp (no babel), how would i
>>>>>> transition from development-mode (20 es-module files) -> production-mode (1
>>>>>> rollup file)?
>>>>>>
>>>>>>
>>>>>> On Fri, May 31, 2019 at 10:47 AM Isiah Meadows <
>>>>>> isiahmeadows at gmail.com> wrote:
>>>>>>
>>>>>>> If it's bundled by Rollup or Webpack into a single bundle, it's
>>>>>>> equivalent to a single `<script type="module" src="...">` pointing
>>>>>>> towards the original entry point, excluding network requests.* But in
>>>>>>> either case, you aren't listing 50 scripts, you're only listing the
>>>>>>> entry module and importing child modules within parent modules.
>>>>>>> Rollup
>>>>>>> and Webpack do mostly the same thing browsers do when it comes to
>>>>>>> resolving dependencies, just they generate a bundle afterwards where
>>>>>>> browsers execute code afterwards. Also, it's worth noting that the
>>>>>>> gap
>>>>>>> between a single large request and multiple smaller requests has
>>>>>>> shrunk a lot since HTTP/2 came along, since it's binary, it allows
>>>>>>> requests and response data to be interleaved, it better leverages the
>>>>>>> underlying TCP protocol format, and it allows servers to send data
>>>>>>> pre-emptively without the client requesting it first. (Web sockets
>>>>>>> are
>>>>>>> built on this functionality.) It's still better to bundle in general,
>>>>>>> but it's less of a problem not to.
>>>>>>>
>>>>>>> This is *not* the case for `<script type="module">` elements - those
>>>>>>> operate more like inline scripts that happen to have the ability to
>>>>>>> `import`.
>>>>>>>
>>>>>>> Oh, and yes, I've loaded upwards of 50-100 modules in development. 20
>>>>>>> modules is *easy* to achieve in single-page apps.
>>>>>>>
>>>>>>> * This is, of course, not the case if you are using pure ES6 and you
>>>>>>> aren't using any plugins to, say, run the original source through
>>>>>>> Babel for React + JSX or something.
>>>>>>>
>>>>>>> -----
>>>>>>>
>>>>>>> Isiah Meadows
>>>>>>> contact at isiahmeadows.com
>>>>>>> www.isiahmeadows.com
>>>>>>> On Sat, May 25, 2019 at 2:12 AM kai zhu <kaizhu256 at gmail.com> wrote:
>>>>>>> >
>>>>>>> > Asynchronous loading differs only in
>>>>>>> > that it takes more code to express the same logic and you have to
>>>>>>> take
>>>>>>> > into account concurrent requests (and you need to cache the
>>>>>>> request,
>>>>>>> > not the result), but it's otherwise the same from 1km away.
>>>>>>> >
>>>>>>> >
>>>>>>> > so async-loading 50 ```<script type="module">``` tags
>>>>>>> > has equivalent side-effect
>>>>>>> > as sync-loading single webpack-rollup (of same 50 modules)?
>>>>>>> >
>>>>>>> > i have nagging suspicion of doubts.  has anyone tried native
>>>>>>> async-loading large numbers (>10) of
>>>>>>> > ```<script type="module">``` tags, and verify it resolves
>>>>>>> identically to using a single webpack-rollup?
>>>>>>> >
>>>>>>> > again, i'm not that knowledgeable on es-modules, so above question
>>>>>>> may be trivially true, and i'm just not aware.
>>>>>>> >
>>>>>>> > -kai
>>>>>>> >
>>>>>>> > On 24 May 2019, at 23:41, Isiah Meadows <isiahmeadows at gmail.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > There's two main reasons why it scales:
>>>>>>> >
>>>>>>> > 1. Modules are strongly encapsulated while minimizing global
>>>>>>> pollution.
>>>>>>> > 2. The resolution algorithm applies the same logic no matter how
>>>>>>> many
>>>>>>> > modules are loaded.
>>>>>>> >
>>>>>>> > It's much easier for it to scale when you write the code unaware of
>>>>>>> > how many modules you might be loading and unaware of how deep their
>>>>>>> > dependency graph is. Fewer assumptions here is key. It's an
>>>>>>> > engineering problem, but a relatively simple one.
>>>>>>> >
>>>>>>> > If you want a short example of how sync module resolution works,
>>>>>>> you
>>>>>>> > can take a look at this little utility I wrote:
>>>>>>> > https://github.com/isiahmeadows/simple-require-loader. That
>>>>>>> doesn't
>>>>>>> > asynchronously resolve modules, but it should help explain the
>>>>>>> process
>>>>>>> > from a synchronous standpoint. Asynchronous loading differs only in
>>>>>>> > that it takes more code to express the same logic and you have to
>>>>>>> take
>>>>>>> > into account concurrent requests (and you need to cache the
>>>>>>> request,
>>>>>>> > not the result), but it's otherwise the same from 1km away.
>>>>>>> >
>>>>>>> > -----
>>>>>>> >
>>>>>>> > Isiah Meadows
>>>>>>> > contact at isiahmeadows.com
>>>>>>> > www.isiahmeadows.com
>>>>>>> >
>>>>>>> > On Thu, May 23, 2019 at 10:49 AM kai zhu <kaizhu256 at gmail.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > actually, i admit i don't know what i'm talking about.  just
>>>>>>> generally confused (through ignorance) on how large-scale es-module
>>>>>>> dependencies resolve when loaded/imported asynchronously.
>>>>>>> >
>>>>>>> > On Wed, May 22, 2019 at 10:42 PM Logan Smyth <
>>>>>>> loganfsmyth at gmail.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > Can you elaborate on what loading state you need to keep track of?
>>>>>>> What is the bottleneck that you run into? Also to be sure, when you say
>>>>>>> async-load, do you mean `import()`?
>>>>>>> >
>>>>>>> > On Wed, May 22, 2019, 20:17 kai zhu <kaizhu256 at gmail.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > i don't use es-modules.
>>>>>>> > but with amd/requirejs, I start having trouble with
>>>>>>> module-initializations in nodejs/browser at ~5 async modules (that may or
>>>>>>> may not have circular-references).  10 would be hard, and 20 would be near
>>>>>>> inhuman for me.
>>>>>>> >
>>>>>>> > can we say its somewhat impractical for most applications to load
>>>>>>> more than 50 async modules (with some of them having circular-references)?
>>>>>>> and perhaps better design/spec module-loading mechanisms with this
>>>>>>> usability concern in mind?
>>>>>>> >
>>>>>>> > p.s. its also impractical for me to async-load 5 or more modules
>>>>>>> without using globalThis to keep track of each module's loading-state.
>>>>>>> > _______________________________________________
>>>>>>> > es-discuss mailing list
>>>>>>> > es-discuss at mozilla.org
>>>>>>> > https://mail.mozilla.org/listinfo/es-discuss
>>>>>>> >
>>>>>>> >
>>>>>>> > _______________________________________________
>>>>>>> > es-discuss mailing list
>>>>>>> > es-discuss at mozilla.org
>>>>>>> > https://mail.mozilla.org/listinfo/es-discuss
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>> _______________________________________________
>>>>>> es-discuss mailing list
>>>>>> es-discuss at mozilla.org
>>>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20190601/194f0d81/attachment-0001.html>


More information about the es-discuss mailing list