ModuleDeclarationInstantiation behaviour after failure

[Allen Wirfs-Brock]

> On Jul 15, 2016, at 7:32 AM, Jon Coppeard <jcoppeard at mozilla.com> wrote:
> 
> Thank you for the detailed response.
> 
> To start with I should say that although I'm working on an
> implementation of the basic module loader defined by the HTML spec for
> <script type="module">, a goal is also to validate that spec.  As it
> stands it suffers from the issue we are discussing and attempts to
> simultaneously load multiple top-level modules but does not correctly
> handle errors in instantiation.

Can you point me at the parts of the HTML spec. that involved here?  ModuleDeclarationInstantiation is a semantics of the ES spec. and it isn’t clear to me why the HTML spec. would need to invoke it. From you description, I suspect there may be some impedance mismatches between the specs.

> 
> In the browser, this simultaneous loading is important for performance
> reasons.  Using modules already has the potential to create many more
> server round trips as dependencies are fetched, so browsers will want to
> parallelise these fetches as much as possible.  This means parsing
> modules as soon as they have been fetched to extract their dependencies
> and starting to fetch their dependencies straight away, all across
> multiple module trees.

Shouldn't be a problem.  Just like some  browsers now  eagerly (or lazily) parse scripts on a separate thread they should be able to parse individual module files as they fetch them and cache parse results (eg, ASTs). Even share cached parse results across multiple windows.  Fetching and mechanism of how a parsed module or script is internally represented is outside of the scope of the ES spec. Eager parsing is explicitly allowed for in the spec. For example, see the NOTE in https://tc39.github.io/ecma262/#sec-toplevelmoduleevaluationjob <https://tc39.github.io/ecma262/#sec-toplevelmoduleevaluationjob>. Basically an implementation can cache or do anything it wants as long as the fact it is doing so is not observable to ES code. The non-observability requirement is the reason it says that error reporting most be deferred until the explicit parse step in the spec. is reached.

> 
> I think it's important to outline how this is going to work in the HTML
> spec itself, especially since the algorithm chosen can have subtle
> effects on the behaviour.
> 
> The approach you suggested above would work but I think it would have
> the disadvantage that while while the transaction for one module tree
> was in progress it would delay modules from a second module tree being
> parsed (and their dependencies fetched).  Did I understand that correctly?

That’s certainly not the intent.  Note that https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule <https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule> is required to be idempotent if it completes normally. An implication of this is that any complete and valid closed subgraph of a module tree should be reusable and sharable among other module trees, even if the first module tree to process it fails because other subtree are incomplete.  Processing multiple module trees in parallel certainly complicates caching and error recovery. But should be possible.  The ES spec. really doesn’t care, as long as the implementation specific quirks aren’t observable to ES code.

(BTW, a loader API that includes the ability of ES code to programmatically introduce additional modules will also complicate eager or parallel processing.  If that is possible, then failure of eager processing of some module trees might be a temporary condition, and failure means that the processing of the module tree needs to be deferred until the API causes something to change.    In other words, statically fetched modules that have dependencies upon dynamically created modules are tricky )

> 
> Another similar approach that also uses the committed state might be to
> parse all modules as they become available and discard any uncommitted
> dependencies on instantiation error.  Dependencies of a top-level module
> are marked as committed on successful load.  I think this would be
> equivalent to some serial execution of TopLevelModuleEvaluationJob for
> each top-level module.

All you really need to throw away are newly created Module Environment Records as they are the entities that are observable to the ES evaluation semantics. But remember to distingish the implementation’s internal data structures from the specification’s abstract data structures such as Module Records, and Module Environment Records. The implementation is free to use any valid memorization it can come up with. 

Note that the possibility of eager linking/instantiation is why module evaluation is a separate step and not included in module instantiation.  The job queue for TopLevelEvaluationJobs is there to enable the host environment to specify the serial execution order of top level module (and script) evaluations.



> 
> However that still has the disadvantage that too many modules are thrown
> away on error.  This means that some modules might have to be re-fetched
> later if they were used by a sequent load, even if they were present
> when that load started.  My concern is mainly that handling this corner
> case would complicate the HTML spec.

Don’t see why refetching would be required (assuming the remote resource doesn’t change).  Might have to relink subtrees that had earlier had linkage failures.
> 
> I guess what I'd really like is some support for this simultaneous
> loading in ES.  Removing the restriction on discarding modules after
> failure and making ModuleDeclarationInstantiation fail if called again
> after failure would be one way.  Another would be to report to the
> caller which modules failed to instantiate.  Either way the problem is
> that it's hard to handle errors from ModuleDeclarationInstantiation if
> you don't know which modules failed.

My sense is that you are seeing requirements/restrictions in the ES spec. that aren't there (or aren’t intended if they are).

We should probably focus on individual issues, one at a time.

Allen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20160715/6abd3754/attachment.html>