Flattening syntactic tail nests (paren-free, continued)

Claus Reinke claus.reinke at talk21.com
Mon Apr 18 11:08:13 PDT 2011

Hi Dave,

thanks for your reply. I was beginning to fear that these
suggestions had been missed. Let me start directly with
your main point, because it biases the detailed evaluation:

> * making nested callbacks better via all of the above
> I think this is a deeper problem that can't be solved with
> cosmetics. This is why I've worked on generators as a
> good solution for the nested-callback problem.

Fortunately, this is a point were I can argue with some
confidence. I like generators and your usage of generators
(it would be nice to have some more written examples of
what one will be able to do, btw), but until you've got yield*
(delegated yield), they won't address the same problem
range. Even then, it depends on the usage.

The general usage scenario I am thinking of is derived
from monadic programming. Depending on where you're
coming from, this is lambda-calculus with added effects
(such as exceptions, threaded state, non-determinism), or
it is simply a refinement of continuation-based programming
(separating the basic operations from composition with
callbacks, by introducing a separate interface for callback
chaining). Either way, we know that the programming
patterns are expressive enough to cover a wide range of
sequential, and some concurrent programming idioms.

This knowledge is based on nearly 20 years of programming
practice, and if you count the use of continuations and monads
in language semantics, you can easily add a decade or two [1].

Since you are working at the borderline toward concurrent
programming (coroutines, asynchronous i/o, non-preemptive,
cooperative scheduling), you think that something more
than syntax is involved. And you're right, in a way, but even
in your usage scenarios, your code will be helped by better
syntax. And you really do not want to emulate monads by
hand-coding yield-based trampolining, even though that
seems useful for async coding and coroutines [4].

The reason why cosmetics will help is that the semantic
problem is already solved, or is in the process of being
solved separately, in your case. Callback nesting does not
only occur in async coding, but arises generally in continuation
based coding styles, including monadic programming.

What all those applications of callbacks have in common
is that they emulate a language that looks like let declarations
(actually let expressions or let statements) but does something
more behind the scenes

    let(a = operation1(..)) {
    let(b = operation2(..)) {

In plain code, this does no error handling, no backtracking,
etc, just bindings followed by evaluating a block. In monadic
code, this could do error handling behind the scenes (if the
handling is fairly standard), or backtracking (eg, if that code
was part of a parser, it might try operation2 for every possible
parse of operation1, until both succeed), or some other effect,
depending on the monad in use. There are whole cottage
industries around monadic programming in various languages,
some aware that they are using monads, others not.

Languages like Haskell and F# take this programming style
seriously and introduce special syntax for it [2,3] (do-notation
in Haskell, since Haskell 1.3; computation expressions in F#).

But the desugaring of the special syntax is based on
callback nesting. Leaving out some details, the let statements
in the example could be desugared to

    (function(a) {
        (function(b) {

Apart from possible side-effects, this is standard lambda-calculus,
or correspondence between parametric and declarative forms, if
you will;-) The nesting is important as it allows us to build up
scopes in which all intermediate results are available.

Callback nesting uses that desugaring, but takes control of the
function applications (they become callbacks, which I'll write as
curried parameters, for clarity)


This is a basic continuation-passing style, which leads to nested
callbacks, and depends on tail-call optimization to work (without
tco, one has to introduce an additional layer of interpretation, to
keep stack usage low by trampolining).

Monadic style simply says that the composition with callbacks
should be separated from the basic operations, so each operation
would implement a simple interface for callback composition.

For the purposes of this overview, there will be some way to
inject values into the monad (Monad.value(val)) and some way
to pass the result of a monadic computation to a monadic
continuation (Monad.then(continuation)). The example changes


but most code can now be written to use the interface (value/then)
instead of any concrete implementation of that interface. In other
words, one can now write reusable code that will work with any
monad. As a special case, one can define monads that combine
effects - for instance, one could have a monad for async i/o and
combine it with a monad for boilerplate error handling. More
generally, the generic monadic library code will include control
structures (such as loops).

Anyway, the semantic basis is there in Javascript, so we mostly
need to clean up the syntax to make these programming patterns
useable. My suggestions were minimalistic, just removing those
unnecessary and distracting nested parens and braces.

Alternatively, one could go the way of Haskell and F#, and
introduce more syntactic sugar, perhaps in the form of monadic
let bindings. Then our example could be written as

    letM(a = operation1(..)) {
    letM(b = operation2(..)) {

(desugaring to the previous form) and we'd also need some
way to state which monad this code is to use.

I just thought it would be appropriate to take the smaller
steps first, just removing the parens and braces, to make
the code structure stand out better

    (operation1(..).then @ function(a) =>
     operation2(..).then @ function(b) =>

Assuming the grammar extensions can be worked out,
this looks like a clear improvement, and it is entirely
syntactic. It doesn't go as far as full monadic syntax
sugar, but is more widely applicable.

Monadic programming is one of the programming idioms
I'd not like to lose, and it is awkward in current Javascript.
Blogs on async coding, error handling, parser combinator
libraries, McCarthy's amb, etc, suggest that many other
Javascript coders are already playing with monadic ideas
(without necessary linking their experiments to monads).

Perhaps ES/next/next will have monadic syntax sugar
as well, following Haskell and F#, but for ES/next I wanted
to start with something less ambitious that will do the job
for a while.

Here are the suggestions in again, mnemonically:

>> 1a (curried definition shorthand)
>>     function (..)(..) { .. } -->  function (..) { return function (..) 
>> { .. }}
>> 1b (function bodies, optional braces)
>>     (function (..)..(..) => ..) --> (function (..)..(..) {..})
>> 2 (function applications, optional parens):
>>     (f @ x) --> f(x)

To address your other comments:

These suggestions are not limited to functions with expression
bodies (though they'd apply there, too, and 1b has been tried
there already) - they simply shift the responsibility for delimiting
the function body or function application one level up, where it
is possible to remove nested parens and braces, because one
ending paren/brace can end several constructs at once.

The use of '@' is unfortunate, but I'm just not sure whether it
will be possible to use just juxtaposition instead (even if it
should turn out to be unambiguous in theory, coders will
find it confusing if there is no explicit delimiter).

The use of '=>' is not bike-shedding - as with '@', it simply
replaces hard-coded parens/braces with an infix marker,
allowing me to make the parens/braces optional in the
grammar, and to remove nested parens/braces in code.

The specific symbols were chosen because the use of '@'
for application is traditional in implementation papers, and
the use of '=>' was the most popular in the list archives.


[1] Reynolds, "The discoveries of continuations", 1993.
    Lisp and Symbolic Computation 6 (3/4): 233-248

[2] Section 7.2 ("Monads") in: Hudak, Hughes, Peyton
    Jones, Wadler, "A History of Haskell: being lazy with
    class", The Third ACM SIGPLAN History of Programming
    Languages Conference (HOPL-III) San Diego, California,
    June 9-10, 2007.

[3] Syme, "Some Details on F# Computation Expressions",

[4] http://www.neilmix.com/2007/02/07/threading-in-javascript-17/


More information about the es-discuss mailing list