typed array strawman proposal

[Kenneth Russell]

Apologies for not replying to the thread properly; just joined the list.

The typed array proposal Vlad sent out covers the WebGL use case and
others which assemble data on the CPU to be handed to, for example, a
graphics or audio subsystem.

I've seen some feedback that the proposal misses an important use case
of parsing binary data read from the network. In such cases the
endianness of the data is defined by the file format. The typed arrays
operate on native-endian data (big-endian or little-endian, depending
on the host). It is possible, though tedious and inefficient, to use
the typed arrays and multiple overlapping views to read and write
multi-byte values (ints, floats) with the correct endianness. People
have pointed to the java.nio.ByteBuffer class as an example API, in
particular its methods to read and write all of the Java primitive
types with the appropriate endianness.

This functionality can be achieved with a new view type, without
perturbing the existing typed array views in any way. A DataArray
class can be added which supports unaligned loads and stores of all of
the primitive types, with a well-defined endianness. Here is some
prototype IDL:

[
  Constructor(in ArrayBuffer buffer,
              optional in unsigned long byteOffset,
              optional in unsigned long length)
]
interface DataArray : TypedArray {
  // Gets the value of the given type at the specified byte offset
  // from the start of the view. There is no alignment constraint;
  // multi-byte values may be fetched from any offset.
  //
  // For multi-byte values, the optional littleEndian argument
  // indicates whether a big-endian or little-endian value should be
  // read. If false or undefined, a big-endian value is read.
  //
  // These methods throw exceptions if they would read beyond the end
  // of the view.
  unsigned byte getUInt8(in unsigned long byteOffset);
  byte getInt8(in unsigned long byteOffset);
  unsigned short getUInt16(in unsigned long byteOffset, [optional] in
boolean littleEndian);
  short getInt16(in unsigned long byteOffset, [optional] in boolean
littleEndian);
  unsigned long getUInt32(in unsigned long byteOffset, [optional] in
boolean littleEndian);
  long getInt32(in unsigned long byteOffset, [optional] in boolean
littleEndian);
  unsigned long long getUInt64(in unsigned long byteOffset, [optional]
in boolean littleEndian);
  long long getInt64(in unsigned long byteOffset, [optional] in
boolean littleEndian);
  float getFloat(in unsigned long byteOffset, [optional] in boolean
littleEndian);
  double getDouble(in unsigned long byteOffset, [optional] in boolean
littleEndian);

  // Stores a value of the given type at the specified byte offset
  // from the start of the view. There is no alignment constraint;
  // multi-byte values may be stored at any offset.
  //
  // For multi-byte values, the optional littleEndian argument
  // indicates whether the value should be stored in big-endian or
  // little-endian byte order. If false or undefined, the value is
  // stored in big-endian byte order.
  //
  // These methods throw exceptions if they would write beyond the end
  // of the view.
  void setUInt8(in unsigned long byteOffset, in unsigned byte value,
[optional] in boolean littleEndian);
  void setInt8(in unsigned long byteOffset, in byte value, [optional]
in boolean littleEndian);
  void setUInt16(in unsigned long byteOffset, in unsigned short value,
[optional] in boolean littleEndian);
  void setInt16(in unsigned long byteOffset, in short value,
[optional] in boolean littleEndian);
  void setUInt32(in unsigned long byteOffset, in unsigned long value,
[optional] in boolean littleEndian);
  void setInt32(in unsigned long byteOffset, in long value, [optional]
in boolean littleEndian);
  void setUInt64(in unsigned long byteOffset, in unsigned long long
value, [optional] in boolean littleEndian);
  void setInt64(in unsigned long byteOffset, in long long value,
[optional] in boolean littleEndian);
  void setFloat(in unsigned long byteOffset, in float value,
[optional] in boolean littleEndian);
  void setDouble(in unsigned long byteOffset, in double value,
[optional] in boolean littleEndian);
};

Note that this view is stateless, and it would be trivial to write a
stream-like class wrapping it. The individual accessors would be very
easy to intrinsify as a few assembly instructions each.

I'd like to suggest that something like this either be mentioned in an
appendix of the TypedArray proposal, or added to the proposal itself.
It will likely cover the majority of the use cases of parsing binary
files efficiently. (How to get the file's contents as an ArrayBuffer
is a separate question.)

Comments?

-Ken