ModuleDeclarationInstantiation behaviour after failure

[Jon Coppeard]

Thank you for the detailed response.

To start with I should say that although I'm working on an
implementation of the basic module loader defined by the HTML spec for
<script type="module">, a goal is also to validate that spec.  As it
stands it suffers from the issue we are discussing and attempts to
simultaneously load multiple top-level modules but does not correctly
handle errors in instantiation.

In the browser, this simultaneous loading is important for performance
reasons.  Using modules already has the potential to create many more
server round trips as dependencies are fetched, so browsers will want to
parallelise these fetches as much as possible.  This means parsing
modules as soon as they have been fetched to extract their dependencies
and starting to fetch their dependencies straight away, all across
multiple module trees.

I think it's important to outline how this is going to work in the HTML
spec itself, especially since the algorithm chosen can have subtle
effects on the behaviour.

The approach you suggested above would work but I think it would have
the disadvantage that while while the transaction for one module tree
was in progress it would delay modules from a second module tree being
parsed (and their dependencies fetched).  Did I understand that correctly?

Another similar approach that also uses the committed state might be to
parse all modules as they become available and discard any uncommitted
dependencies on instantiation error.  Dependencies of a top-level module
are marked as committed on successful load.  I think this would be
equivalent to some serial execution of TopLevelModuleEvaluationJob for
each top-level module.

However that still has the disadvantage that too many modules are thrown
away on error.  This means that some modules might have to be re-fetched
later if they were used by a sequent load, even if they were present
when that load started.  My concern is mainly that handling this corner
case would complicate the HTML spec.

I guess what I'd really like is some support for this simultaneous
loading in ES.  Removing the restriction on discarding modules after
failure and making ModuleDeclarationInstantiation fail if called again
after failure would be one way.  Another would be to report to the
caller which modules failed to instantiate.  Either way the problem is
that it's hard to handle errors from ModuleDeclarationInstantiation if
you don't know which modules failed.

Jon

On 14/07/2016 17:20, Allen Wirfs-Brock wrote:
> 
>> On Jul 14, 2016, at 3:51 AM, Jon Coppeard <jcoppeard at mozilla.com
>> <mailto:jcoppeard at mozilla.com>> wrote:
>>
>> On 07/07/2016 22:33, Allen Wirfs-Brock wrote:
>>> I would expect implementations to
>>> discard any module records it created during a linking phase that throws
>>> errors.
>>
>> I think it's not trivial to know which module records to discard.
>>
>> Thinking out loud:
>>
>> A module loader may be simultaneously loading multiple top-level modules
>> which have overlapping dependency graphs.  So an imported module may be
>> created to satisfy the dependencies of more than one top-level module
>> that is currently being loaded.
> 
> In the spec, module instantiation is always initiated by a
> TopLevelModuleEvaluationJob. Such jobs are serially executed to
> completion.  The spec. explicitly says that an implementation may
> pre-instantiate and link modules ( it could even happen during a build
> process) as long as error reporting is deferred to the actual
> TopLevelModuleEvaluationJob (of course a build time linker would also
> report build time errors).
> 
> So, form the spec perspective there is no semantics of “simultaneously
> loading” multiple top-level modules.  To conform to those semantics, an
> implementation that wants to simultaneously eagerly instantiate and link
> modules has to do it in a manner that preserves the serial instantiation
> semantics.
> 
> 
>>
>> We can't throw away all dependencies on failure because that could
>> discard successfully loaded modules that are in use by a previously
>> loaded module.  A subsequent load of another module could then re-load a
>> different version of these modules.
>>
>> Really we only need to throw away the module which failed to link and
>> every ancestor module up to the root of the tree, i.e. those for which
>> instantiation has started but did not complete.  But a module loader
>> doesn't have a way to work this out.
>>
>> One solution might be to have the loader maintain an 'instantiated' flag
>> for each module which is initially false and to set this for all
>> uninstantiated descendants on successful instantiation.  Then we could
>> throw away uninstantiated descendants on failure.  (This would discard
>> more than necessary, but that doesn't matter).
>>
>> Does that make sense?  I think this would work but it feels like it's
>> making the loader do extra work to compute state that could more simply
>> be stored in the module record itself.
> 
> I’m just thinking out load too,  But here is how I would approach it.
> 
> Module Records are created by ParseModule (often indirectly via
> HostResolveImportedModule) and subsequently retrieved by
> HostResolveImportModule. 
> 
> I would treat steps 3-5 of
> [TopLevelModuleEvaluationJob](https://tc39.github.io/ecma262/#sec-toplevelmoduleevaluationjob)
> as an atomic transaction that either succeeds or fails.  During such a
> transaction, any new module record that is created is considered a
> “pending module record”. When a transition successfully completes, all
> of its pending module records permanently become “committed module
> records” but if a transaction fails its pending module records are
> discarded.  During a transaction, HostResoveImportedModule uses both the
> committed and pending module record sets to resolve import requests.  
> 
> Regarding storing state (for example, committed or pending) in module
> records:  Module records (like most data structures in the spec) are
> just abstrations used by the specification to describe the semantics and
> observable state changes.  An implementation is free to represent those
> abstractions any way it wants and certainly can incorporate additional
> unobservable implementation state.
> 
> Allen
> 
> 
>