Array subclassing, .map and iterables (Re: Jan 30 TC39 Meeting Notes)

[Claus Reinke]

Thanks for the explanations and additional details. Let me first try 
to rephrase, to see whether I've understood your reasoning:

The problem comes from the partial integration of types in ES, 
specifically having typed arrays but no easy way to express and 
control the types of the functions mapped over them.

And your solution is to fix Array.map to being type-preserving, and 
to use an auxiliary map in Container.from instead of Array.map 
when type changing mappings have to be expressed. 

Using <> for type parameters, => for function types, and suppressing 
a few details (such as prototype, this, index parameter), we can write 
the types of the two groups of operations as

    Array<Elem>.map : 
        (<Elem> => <Elem>) => 
        Array<Elem>

    Container<Elem2>.from : 
        Iterable<Elem1> => 
        (<Elem1> => <Elem2>) => 
        Container<Elem2>

where the ES5 Array is seen as Array<Any> (so arbitrary mappings 
are still supported at that level), and Array<Int32>, etc are written 
as type-level constants Int32Array, for lack of type-level constructor 
syntax (the parameterized Interface Iterable<> is also inexpressible).

Since ES cannot guarantee that the mappings have the expected
types, an implicit conversion of the mapped elements to the 
expected element type will be enforced (probably with a type
check to avoid unexpected conversions?).

So 

    int32Array.map( f ) 

will be read as roughly

    int32Array.map( (elem) => Number( f(elem) ) )

and

    Int32Array.from( iterable, f )

as

    Int32Array.from( iterable, (elem) => Number( f(elem) ) )

Do I have this right, so far?

> var intArray = new Int32Array([42,85,127649,32768]); 
> //create a typed array from a regular array
> var strArray = intArray.map(v=>v.toString());
> 
> If intArray.map() produces a new intArray then the above map 
> function is invalid.  If intArray.map() produces an Array instance
> then you intArray.map instance of intArray.constructor desire 
> won't hold.  We can't have it both ways without provide some
> additional mechanism that probably involves additional 
> parameters to some methods or new methods. 

It is this additional mechanism which I'm after. In typed languages,
it is long-established practice to put the additional parameters at
the type level and to hang the interface on the type-level constructor,
and I wonder how much of that could be translated for use in ES.

For instance, being able to specify an overloaded map function 
was the motivating example for introducing type constructor
classes in Haskell

    A system of constructor classes: overloading and implicit 
    higher-order polymorphism
    Mark P. Jones, 
    In FPCA '93: Conference on Functional Programming Languages 
    and Computer Architecture, Copenhagen, Denmark,  June 1993.
    http://web.cecs.pdx.edu/~mpj/pubs/fpca93.html

> 1) Array.prototype.map produces the same kind of array that it 
> was applied to, so:
> 
> for the above example
>   m instance of V will be true.   
>   intArray.map(v=>v.toSring()) produces an Int32Array.  
> The strings produced by the map function get converted back to numbers.

Given the history of implicit conversions in ES, have you considered 
just doing runtime type checks, without those new implicit conversions?

> 2) If you want to map the elements of an array to different kind of array 
> use <ArrayClass>.from with  a map function as the second parameter:
> 
> var strArray = Array.from(intArray, v=>v.toString());
> 
> This seemed like a less invasive change then adding additional target kind
> parameters to Array.prototype.map.  Also it seems like a very clear way for
> programmers to state their intent.
  
> ES isn't Java or C#.  We don't have formalized interfaces (although it 
> is useful to think and talk about informal interfaces) and since we are
> dynamically typed we don't need to get sucked into the tar pit of generics.

If a programming concept is as useful as interfaces are, it usually pays 
to think about language support for it. And I was certainly not thinking
of Java or C#, more of TypeScript, where the team seems to be working
on JavaScript-suited generics for the next version, to be able to type
current JavaScript library code. 

Btw, parametric polymorphism in ML and its refinements and
extensions in Haskell were elegant and concise tools before they got 
watered down in a multi-year process to fit into Java. If you have bad
experience with generics, they probably come from Java's adaption.

> How would use produce an Array of strings from an Int32Array?

Somewhat like

    Array.from( int32Array ).map( (elem) => elem.toString() )

Implementations would be free to replace the syntactic pattern
with an optimized single pass (in more conventional optimizing
language implementations, such fusion of implicit or explicit loops 
is standard, but even ES JIT engines -with their limited time for 
optimization- should be able to spot the syntactic pattern).

>> - instead of limiting to Array, .from-map is now limited to iterables
>>   (it would work for Set, which is really OrderedSet, but it wouldn't
>>   work for WeakMap)
> 
> We already have Array.from that works with iterables, how does 
> adding a map function change anything related to the 
> <ArrayClass>.from result domains    

My point was that map is far more widely useful, not limited to
Array (Array.prototype.map), and not limited to Array construction
from Iterables (Array.prototype.from with second parameter). 
Consider map on event emitters, on promises, on exceptions, 
on generators, ..

I don't have an alternative solution that would cover all use cases 
in ES uniformly, because the existing solutions in other languages
do not translate directly. 

However, I wanted to ring a warning bell that adding a different 
partial solution for every new use case is not going to scale well 
(especially with things being so difficult to change once they are 
in ES), and misses potential for writing generic library code. 

If I have to write different code, depending on whether I need 
to map over a constructed Array, an Array under construction, 
an Array or an Int32Array, a generator, a promise, etc., then 
that will result in duplicate code instead of generic code.

>> With a general solution to the issue, I would expect to write
>> 
>>   SubArray.from( iterable ).map( val => val ) instanceof SubArray
> 
> yes, the above will produce an instance of SubArray.  But the 
> above also has the cost of an extra copy and the map function
> doesn't get to see the original iterable's values. 

Implementations can fuse .from().map() as well as .map().from();
if you want access to the unconverted elements, you want to map
over the iterable, not the resulting Array (again, with loop fusion
in the implementation):

    SubArray.from( iterable.map( val => f(val) ) )

Of course, since map isn't part of a standard Array-independent
interface, I have to write that as a generator expression (not sure
whether I'm up to date on their syntax) instead of a map.

    SubArray.from( (for ( elem of iterable ) in f(elem) ) )

>> while also getting
>> 
>>   new Mapable().map( val => val ) instanceof Mapable
> 
> I don't even know how to interpret the above, as we don't have 
> a class or constructor named Mapable.

fair enough - Mapable being intended as an interface for classes
implementing map wouldn't be new-able, so this was pseudo code
for instantiating any class that implements such a Mapable. I think 
a full solution for polymorphic map will require some way of 
specifying interfaces independent of the classes (Array, Int32Array, 
Promise, generators, ..) implementing them.

>> PS. What about array comprehensions and generator expressions?
> 
> What about them?  Array comprehensions are a for of Array 
> initializer and always produce an Array instance. Generaltor 
> expressions produce iterators (which are iterable). 

Array comprehensions are a full array processing language, 
which can be desugared to explicit array operations like map
and filter, so they tend to have the same problems as map
and filter do. Will I be able to use array comprehensions with 
Int32Arrays, or to convert between array types? Will I be able
to call .map on iterators/generator expressions, without using
case-specific syntax?

Claus