Proposal: `await.all {...}` for parallelism

Tom Boutell tom at apostrophecms.com
Mon Nov 25 12:55:16 UTC 2019


Hey, you're absolutely right! It's OK because it just means things are more
deterministic before the block exits. It doesn't impact any reasonable
expectations *during* the block.

I am convinced that your syntax is useful and does not introduce any new
confusion.

I wonder, then, if it is also possible to implement concurrency limits for
this properly?

await.all({ concurrency: 5 }) {
  for (const item of items) {
    // returns promise
    item.process();
  }
}

This is more challenging because, in our transpilation, we can't just
bottle up all the promises and call Promise.all at the end. It would be too
late to manage how many are in process at once, bashing on various API
limits (:

On Sun, Nov 24, 2019 at 7:43 PM Naveen Chawla <naveen.chwl at gmail.com> wrote:

> Hi! It does not change the meaning of the ";" at all. As you may already
> know, omitting `await` already invokes multiple async function calls in
> parallel, in current JavaScript, so absolutely no change in that respect.
>
> The only thing this `await.all` suggestion does, is ensure that all
> non-awaited async function calls are completed before proceeding beyond the
> end of the block.
>
> i.e. it adds fairly straightforward and terse deterministic control to
> otherwise non-deterministic code, without requiring knowledge of
> destructuring or `Promise.all`.
>
> On Sat, 23 Nov 2019 at 13:25, Tom Boutell <tom at apostrophecms.com> wrote:
>
>> This is very interesting, but this code:
>>
>> await.all {
>>    x = getXAsync();
>>    y = getYAsync();
>> }
>>
>> processXAndY(x, y);
>>
>> Still carries within it the problem that if I'm looking at just the
>> middle of the { ... } block — if "await.all" has scrolled offscreen — I'll
>> be completely wrong about what ";" means. I think that's too much magic.
>>
>> Also, in the case of the "for" loop, this doesn't address managing the
>> level of concurrency. Although it could in theory with a syntax like
>> await.all({ concurrency: 5 }), I'm not sure if it's practical to implement
>> that for your general case.
>>
>> Actually I'm curious about what the implementation would look like in
>> general.  If it were babel compiling this, I guess it would have to wrap
>> every statement not preceded by "await" with a check for whether it returns
>> a thenable and add it to an array if it does. But with the concurrency
>> feature it would also have to defer executing the code at all until the
>> right time as otherwise we're still starting zillions of "processes" at
>> once.
>>
>>
>> On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <naveen.chwl at gmail.com>
>> wrote:
>>
>>> However, if `await.all { ... }` were to mean "wait for all non-awaited
>>> async function calls made within this block to complete before proceeding",
>>> as I suggested earlier, I think that could satisfy determinism for "await"
>>> wherever it is used, and satisfy the original motivation:
>>>
>>> ```
>>> await.all {
>>>     for (const item of items) {
>>>         doTheThingAsync(item);
>>>     }
>>> }
>>> ```
>>>
>>> Notice I have omitted `await` inside the loop. Like current JavaScript,
>>> that causes parallel execution, so no change on that front, from a
>>> determinism perspective. So determinism is not hurt by `await.all`. Rather,
>>> it guarantees completion before going further.
>>>
>>> In an earlier example (paraphrase-coded as I forgot the names):
>>>
>>> ```
>>> let x, y;
>>>
>>> await.all {
>>>    x = getXAsync();
>>>    y = getYAsync();
>>> }
>>>
>>> processXAndY(x, y);
>>> ```
>>>
>>> I think the benefit of this syntax appears more stark with the looped
>>> (first) example, as current JavaScript requires building an array in the
>>> loop to subsequently pass to `Promise.all`, which I think is a little more
>>> difficult to conceptualize than the `await.all { ... }` way of doing it.
>>> The 2nd example is arguably better than current JavaScript too,
>>> particularly because the coder doesn't have to be very smart with
>>> destructuring in light of understanding the "Promise.all" return type, etc.
>>> In other words, less cognitive overhead, which I think is a net positive.
>>>
>>> On Fri, 22 Nov 2019 at 13:44, Tom Boutell <tom at apostrophecms.com> wrote:
>>>
>>>> I am very sympathetic to pitches to allow more common cases for promise
>>>> libraries to be written in an "awaitful" syntax without thinking explicitly
>>>> about promises.
>>>>
>>>> Howeever I think that changing the meaning of the semicolon in a
>>>> particular context has too much potential for confusion. As others have
>>>> said, parallel execution is different, and it should look and feel
>>>> different. The most basic assumption a developer makes (consecutive lines
>>>> of code run consecutively) is difficult to get away from; that's why we
>>>> introduced "await" in the first place, to bring back the ability to write
>>>> deterministic code with consecutive statements. Which sounds like a
>>>> reasonable ask, when it's put that way. (:
>>>>
>>>> I did propose this recently:
>>>>
>>>> for (const item of items concurrency 5) {
>>>>   await  doTheThing(item);
>>>> }
>>>>
>>>> However in this case I'm not talking about consecutive statements, I'm
>>>> only talking about rules for simultaneously (in the sense of async, not
>>>> threads) running more than one instance of the block. So I'm not proposing
>>>> that we change the meaning of the semicolon(s) *within* the block in a way
>>>> that could mean that if you're looking at half the code in the middle you
>>>> would be likely to fundamentally misunderstand its operation.
>>>>
>>>> I think that risk - that you can't tell what a semicolon means without
>>>> reference to the outer context - is what makes your proposal a bridge too
>>>> far for me.
>>>>
>>>>
>>>> --
>>>>
>>>> THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
>>>> APOSTROPHECMS | apostrophecms.com | he/him/his
>>>> _______________________________________________
>>>> es-discuss mailing list
>>>> es-discuss at mozilla.org
>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>
>>>
>>
>> --
>>
>> THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
>> APOSTROPHECMS | apostrophecms.com | he/him/his
>>
>

-- 

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20191125/b62eaa73/attachment.html>


More information about the es-discuss mailing list