Native Tensor support
isiahmeadows at gmail.com
Sun Jan 28 18:33:53 UTC 2018
Inline, so it's a little easier to read.
me at isiahmeadows.com
Looking for web consulting? Or a new website?
Send me an email and we can get started.
On Sun, Jan 28, 2018 at 8:27 AM, Robert Eisele <robert at xarg.org> wrote:
> Q1: That's a good question. To me, tensors could have their own habitat
> within the JS scope, without handling existing data types, since they
> typically serve different purposes.
In practice, the data storage requirements are similar, and you could
implement the tensor transforms in terms of static functions over
typed arrays in the first place.
> Q2: I don't quite get what you mean by kernel-like API. Could you go into
> more detail on that?
I was talking in reference to [GPU compute kernels], which are
basically like only specifying the inner part of a `for` loop, but
also without the ability to `break` early. (This is an
over-simplification, but it should help you at least get an idea what
> C1: Sure, that is what I said. It's easy to construct multi-dimensional
> arrays already, but neither is there a shortcut for such arrays, like in C++
> with "new int", nor is there a fast and parallel way of working
> with data in a certain dimension.
In JS, you could create a wrapper class to model typed array data in a
multi-dimentional context, and in fact, that's how C/C++ (and most
other languages with native support) do it under the hood.
However, you're right in that there's no realistic parallel method of
working with data. There *exists* [one runtime] that has decent
parallel threading support (based on Webkit), but that only covers
low-parallelism needs that just need enough parallelism to not be
I/O-bound or blocked on complex CPU computation elsewhere when
> C2: Machine learning is just a momentary hype. Who knows what it will be in
> the next 5-10 years - nano*, quantum*, ... Whatever it is, the trend goes
> into the direction of calculations of high-dimensional data - and another
> trend is that all that will eventually land in the browser. So tensors as an
> abstract algebraic object right at the finger tips of a JS developer could
> facilitate all that.
Tensors aren't necessarily the ideal way to handle it, though. For
example, a lot of high-parallelism needs, such as image processing and
game rendering, need a more general framework, and they may find
themselves needing to implement their own primitives. (For one, it's
possible to do parallel sorting on the GPU with major speedups.)
> I'm not sure if tensors are the best abstraction for DOM parallelism.
> Furthermore, I think static changes should not go beyond 2- or 3 dimensional
> spaces. And as Boris said, it only works if one can make sure there are no
1. They aren't. (I would really need a DOM API for async batch
updates, but that's a whole different story.)
2. You could in theory go into the hundreds of dimensions, but GPUs
have hardware limitations above a certain limit, so the runtime would
have to chunk accordingly.
3. I'm aware. (More accurately, overlaps *or* dependencies, the latter
is much easier to encounter.)
> Am 27.01.18 um 04:49 schrieb Isiah Meadows:
>> Two questions:
>> 1. Would APIs that operate on existing data types (rather than tensors)
>> work just as well?
>> 2. Could there instead be a kernel-like API that could work on things
>> independently? Lower level APIs that enable equivalent high-level constructs
>> is a much better place to start.
>> And two comments:
>> 1. You could easily emulate 2D arrays by simply using 1D arrays and
>> storing data in row- or column-major order. This is pretty well-known at
>> this point, and is how C/C++ allocate multi-dimensional arrays internally.
>> 2. Data parallelization requires special consideration, and I can assure
>> you, machine learning isn't the only thing that could stand to benefit from
>> this. It needs to be broad enpugh that other non-data applications can
>> benefit from it. (Applying DOM changes from a static change list is an
>> embarassingly parallel\* problem. So anything that could speed this up by a
>> substantial bit could be infinitely useful to anyone using Angular, React,
>> Ember, or any other framework out there.)
>> \* Yes, that's a technical term. Look it up on Wikipedia.
More information about the es-discuss