Revisiting Decimal

Brendan Eich brendan at mozilla.com
Thu Jan 15 11:52:04 PST 2009


On Jan 15, 2009, at 10:46 AM, Kris Zyp wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Brendan Eich wrote:
>> On Jan 15, 2009, at 9:47 AM, Kris Zyp wrote:
>>
>>> Where is the loss coming from?
>>
>> Decimal-using peer S1 encodes
>>
>> {p:1.1, q:2.2}
>>
>> Double-using peer C1 decodes, adds, and returns
>>
>> {p:1.1, q:2.2, r:3.3000000000000003}
>>
>> The sender then checks the result using decimal and finds an error.
>> Meanwhile the same exchange between S1 and decimal-using peer C2
>> succeeds without error.
>>
>> /be
>>
> Exactly, C1 introduces the error by it's use of binary.

This is not about blame allocation. The system has a problem because,  
even though JSON leaves numeric representation unspecified, higher  
layers fail to agree. That could be viewed as a JSON shortcoming, or  
it could be the fault of the higher layers. I don't want to debate  
which is to blame here right now (more below).

The point is that all the JS self-hosted JSON implementations I've  
seen, and (crucially) the ES3.1 native JSON implementation, use  
double, not decimal. This constitutes an interoperation hazard and it  
constrains compatible future changes to ES-Harmony -- specifically, we  
can't switch JSON from double to decimal by default, when decoding  
*or* encoding.


> The JSON
> encoding didn't introduce the error. JSON exactly represented the data
> it was given,

JSON the RFC is about syntax. It doesn't say more than "An  
implementation may set limits on the range of numbers" regarding  
semantics of implementations.

Actual implementations that use double and decimal will not  
interoperate for all values. That means "not interoperate".


> and the decimal decoding and encoding peer refrains from
> introducing errors as well.

Assuming no errors is nice. (What color is the sky in your world? :-P)  
Meanwhile, back on this planet we were debating the best way to reduce  
the likelihood of errors when adding decimal to JS. Back to that debate:


> Are you arguing that all JSON interaction relies on other peers
> introduces errors according to binary floating computations? I already
> disproved that idea by pointing out there are existing implementations
> that use decimal and thus don't add such errors.

You didn't prove or disprove anything, you hand-waved. For all you or  
I know, and Bob Ippolito agrees, there are latent, hard-to-find bugs  
already out there.

Our bone of contention specifically was making (typeof 1.1m ==  
"number"), and I am certain that this increases the likelihood of such  
bugs. It cannot reduce the likelihood of such bugs. Whether it means  
more work for Dojo and other frameworks that would need to adapt to  
the addition of decimal to a future ES spec is secondary, or really  
just TANSTAAFL.


> If you are arguing
> that there are certain client-server couples have become dependent on
> these errors there are couple faults in this logic. First, the error
> dependencies experienced in languages are almost always going to be
> internalized by devs. Because there may be a number of internal error
> expectations in existence does not imply a significant number of
> inter-machine rounding error expectation dependency.

To the extent I understand what you mean here, I can only disagree --  
completely!

You're arguing by assertion that rounding errors due to double's  
finite binary precision, which are the most reported JS bug at https://bugzilla.mozilla.org 
, are somehow insignificant when JSON transcoding is in the mix.  
That's a bold assertion.


> Second, I am not
> asserting that JSON decoding should automatically convert JSON numbers
> to binary, only that JSON encoding should serialize decimals to
> numbers.

What should JSON.parse use then, if not double (binary)? JSON.parse is  
in ES3.1, and decimal is not.


> In your example, if a server is sending these JSON values to
> C1 ES-harmony based peer for computation, the computations will still
> take place in binary,

Ok, hold that thought.


> unless we were to add some mechanism for
> explicitly specifying how numbers should be decoded.

Let's say we don't. I showed interoperation failure if two peers, C1  
and C2, fail to decode to the same numeric type. You now seem to be  
agreeing that binary should be the one type JSON number syntax decodes  
into. Great, except you want encoders to stringify decimal literals as  
JSON numbers.

This means 1.1m and 1.1 both encode as '1.1' but do not compare as ==  
or === (whatever happens with typeof and the typeof/==/===   
invariant). It also means

123345678901234567890.1234567890123m

encodes as

'123345678901234567890.1234567890123'

but decodes as binary

123345678901234570000

which breaks round-tripping, which breaks interoperation.

Am I flogging a dead horse yet?

Long ago in the '80s there was an RPC competition between Sun and  
Apollo (defunct Massachusetts-based company, but the RPC approach  
ended up in DCE), with both sides attempting to use open specs and  
even open source to build alliances. Bob Lyons of Sun argued  
eloquently for one standard type system for senders and receivers.  
Paul (I forget his last name) of Apollo argued for "receiver makes it  
right" to allow the funky non-IEEE-754 floating point formats of the  
day to be used at the convenience of the sender. E.g. Cray and DEC VAX  
senders would not have to transcode to the IEEE-754 lingua franca,  
they could just blat out the bytes in a number in some standard byte  
order.

Bob Lyon's rebuttal as I recall it was two-fold: 1. "receiver makes it  
right" is really "receiver makes it wrong", because the many types and  
conversions are a fertile field for bugs and versionitis problems  
among peers. 2. There should be indeed be a lingua franca -- you  
should be able to put serialized data in a data vault for fifty years  
and hope to read it, so long as we're still around, without having to  
know which sender sent it, what floating point format that obsolete  
sender used, etc.

The two points are distinct. Complexity makes for versionitis and bug  
habitat, but lack of a single on-the-wire *semantic* standard makes  
for a Tower of Babel scenario, which means data loss. Fat, buggy, and  
lossy are no way to go through college!

See http://www.kohala.com/start/papers.others/rpc.comments.txt for more.

We are not condemned to repeat history if we pay attention to what  
went before. JSON implementations in future ES specs cannot by default  
switch either encoding or decoding to use decimal instead of number.


> JSON does not follow the path of other formats that attempt to dictate
> tight language-type couplings. In all cases, peers can ultimately
> choose how they want to internally handle the data provided by JSON.
> JSON is pure data format, not computation prescription and won't
> dictate how computations are performed.

Yeah, yeah -- I know that and said as much. The argument is not about  
JSON as a syntax spec, it's about what we do in the implementation in  
ES3.1, where we have to make semantic choices. Including types and  
typeof, including future proofing.

/be



More information about the Es-discuss mailing list