Improved syntax for observable mapping and subscribing

Isiah Meadows isiahmeadows at
Sat Mar 24 07:26:11 UTC 2018

Hindsight, I probably sent that last message prematurely.


Here's some of the things I learned while making that, among dabbling
with other things:

The problem domain of working with collections is hard. Generically
modifying them is also hard. It wasn't until we got OO until we
learned how to iterate an immediate collection (sets, arrays, maps).
With interfaces, synchronous iteration\* becomes easy, since you can
only iterate a collection all at once, and there's a predetermined
order that's easy to configure.

- Arrays can be easily iterated in either direction, by simply moving
a pointer where you want it.
- Sets and maps are a bit more complex, but they still have all the
values immediately ready.
- Generators are also a little more complex, but this is where the
interface makes things easy.

Really, all you need for sync iteration is:

- A `.next()` method that returns either "done" (with optional value)
or "not done" (with required value).
- A method to create a collection of that type from a list (if you
plan to allow mapping).

Now, with async stuff, it's all nice to support iteration more
generally, but there's a few other things that need to be included:

- A way to initialize lazy values (lazy properties), collections
(Lodash wrappers, observables), etc.
- A way to break iteration.

Both of these are required for sync iterators already (think:
`.next()` + `.return()`, where `.next()` also forcibly initializes the
collection), but with async collections, it's not always as simple as
that. With traditional `.next()`-based iterators, you can just stop
requesting subsequent values (`.return()` is really just best-effort
notification of "not needed" for resource cleanup), but with
observables, you have to have an explicit notification mechanism in
place, so it knows to stop sending you things and so you know to start
ignoring what you receive.

There's another issue, too: sync iterators can't do things between
iterations, but async ones *could*. The obvious result would be to
allow the user to specify whether to handle the results in parallel,
but not everything supports that (like async coroutines). Also, the
user can't always handle results as quickly as the "iterator" produces
them. This frequently is the case with observables listening to spammy
event emitters, and is why RxJS has `.debounce(ms)`.

Now that we've gotten that out of the way, what now? Well...what about
mapping? How do we go from a bunch of As to a bunch of Bs? This is
where procedural OO starts to falter:

- We find ourselves building a set of Bs as we iterate the As, and we
return that. That's a bunch of boilerplate.
- We can't generically create that set of Bs without knowing the type
of As we have. Interfaces only type *values*, not *types*.

We instead need to look to the functional and type-level stuff, where
this once again becomes easy.

- A `.map(f)` method, which for each `foo` in it, calls `f` with it,
and returns a new collection with all the results.

That is magical. It simplifies a lot. We don't have syntax to capture
it yet in JS, but it's a natural extension of what we already have for
promises. In fact, promise `.then` isn't far off, except there's a
catch: we also need a way to flatten a structure. This is where we
need a new method, to go from a collection of As to a collection of Bs
while transforming a single A to a collection of Bs.

- A `.chain(f)` method, which for each `foo` in it, calls `f` with it,
and returns a new collection with all the values within the results.

Now, this seems very convenient, and it's very well typed. But this
isn't very incredibly convenient: we might only want to flatten it
sometimes, and promises are a good example of this (it's why `.then`
implicitly unwraps promises). A common solution in the functional
world is to always flatten (giving rise to a `.constructor.of`), and
it simplifies it on paper, but it doesn't simplify it for the user.
Instead, we could use two methods to do this, that are a little looser
in their constraints:

- A `.lift(f)` method, which for each `foo` in it, calls `f` with it,
and returns a new collection with all the results, optionally
- A `.chain(f)` method, which does the same, but `f` instead returns
either a wrapped instance or an array of 0 or more values to be

This allows types to more easily be mapped over, but there's a
critical thing this version of `.chain` enables us to do: it allows us
to filter and otherwise manipulate the collection, without being
dependent on the caller's type (it doesn't even need to have a
`.constructor` property). Filtering would equate to `.chain(x => f(x)
? [x] : [])`. If you read the proposal, I wrote a basic implementation
of the `.distinctBy(func)` operator commonly found in observable

This does not address the issue of executing lazily allocated
pipelines, but that's required for iteration. If there's nothing to
lift over, there's conceptually no need to iterate to fulfill the
required interface contract. This necessitates a new operation:

- An `.each(f)` method, which initializes the pipeline and calls `f`
with each result, breaks if `f` returns falsy, and optionally returns
a promise resolved when done.

This enables iteration as well as breaking early. This doesn't cover
the scenario when you want to break from outside the loop (think:
external consumer closed abruptly. `.return()`/`.throw()` on
iterators), but it does cover the scenario of breaking within the

Or to put it another way, iterables are hard, mapping is harder, and
chaining, it's complicated. There's just so many edge cases I could
write a book about it (and I'm pretty sure there already *has* been
one written already, just from the common nature of the topic).

Or to put it another way, pipelines are hard. Looping is hard.
Language design is hard.

Or to put it another way, I have no clue what the hell I'm doing, and
I'm just trying to figure it out as I go along. Diving head-first into
fully opaque water is always fun. :-)

On Fri, Mar 23, 2018 at 7:55 PM, Isiah Meadows <isiahmeadows at> wrote:
> I've already looked into this kind of thing myself privately. I came
> up with this last year [1], and I more recently came up with this [2]
> (the second is much better IMHO). Both of those offer solutions to
> this problem, and they do in fact offer ways for 1. iteration, and 2.
> mapping/filtering/etc.
> [1]:
> [2]:
> -----
> Isiah Meadows
> me at
> Looking for web consulting? Or a new website?
> Send me an email and we can get started.
> On Fri, Mar 23, 2018 at 11:06 AM, Thomas Grainger <tagrain at> wrote:
>> You can convert an observable into an async iterator. You have to choose
>> between discarding or buffering uniterated items
>> On 23 Mar 2018 14:39, "Bob Myers" <rtm at> wrote:
>>> Could someone jog my memory about proposals for better syntax for
>>> observable mapping and subscribing, if any?
>>> I'm getting really tired of writing
>>> ```
>>> foo$.pipe(map(bar => mapper(bar)))
>>> ```
>>> I would much prefer to write something along the lines of
>>> ```
>>> stream function fooMapper(foo$) {
>>>   while async (const bar = foo$()) {
>>>     emit mapper(bar);
>>>   }
>>> }
>>> ```
>>> Yes, I'm aware of all the potential issues here and this is just an
>>> example, not an actual syntax proposal. I'm just wondering about any prior
>>> art.
>>> Bob
>>> _______________________________________________
>>> es-discuss mailing list
>>> es-discuss at
>> _______________________________________________
>> es-discuss mailing list
>> es-discuss at


Isiah Meadows
me at

Looking for web consulting? Or a new website?
Send me an email and we can get started.

More information about the es-discuss mailing list