Thoughts on IEEE P754

Sam Ruby rubys at intertwingly.net
Fri Aug 22 15:49:32 PDT 2008


On Fri, Aug 22, 2008 at 5:46 PM, Mark S. Miller <erights at google.com> wrote:
> On Fri, Aug 22, 2008 at 11:39 AM, Waldemar Horwat <waldemar at google.com> wrote:
>> Sam Ruby wrote:
>>> When dealing with currency, having 0.05 + 0.05 produce 0.10 is a feature.
>>
>> Contrary to some beliefs, the different cohort members in IEEE P754 are *not* a valid indicator of precision.  To prove this, assume that the different cohort members are an indicator of precision.  Then evaluate:
>>
>>  3.00 + 1.00000
>>
>> IEEE P754 returns 4.00000, while the result is actually precise to only two digits after the decimal point.  Hence a contradiction.  QED.
>
> This example makes me realize how utterly confused and ignorant I am
> about the rationale for decimal.

I do know that if I have $5 and add a dime, I expect to have $5.10, and not $5.

I do believe that the folks who worked on IEEE P754 worked through a
lot of scenarios, and can provide exquisite detail for the rationale
for all of these operations.

Yes, I understand that many of these results seem foreign to those of
us (myself included) who have been poisoned by years of only having
binary64 floating point available to us.  I will say that when faced
with artificial examples, my intuition tends to fail me until I find
corresponding examples using currency.

> In particular, on one of the ES3.1 calls, Sam explained why he thought
> mixed mode arithmetic should convert to binary -- IIUC, in order to
> faithfully represent our ignorance of the precision that should be
> associated with the result. From what I thought I'd understood, I
> would have expected the above example to produce 4.00. However, the
> above example makes clear that decimal by itself, even when non-mixed,
> does not attempt to accurately (or conservatively) represent one's
> ignorance of precision.

When talking about decimal128, I believe that 5 is exact, and 0.10 is
exact, and the result of adding the two should also be exact.

When talking about binary64, clearly 5 is also exact, but 0.10 can not
be represented exactly.

> Sam, did I get your rationale wrong? Now that we've seen these
> examples from Waldemar, could you try explaining again why mixed
> should result in binary?

I find it easier to talk about real examples than abstractions.  I've
done the following quickly, so forgive me if I get some detail wrong.

A binary floating point number has 52 bits of fraction, and by
assuming an implicit leading one, they get an additional bit.  This
means that 1.1 is stored as (for brevity, I'll use hex)

[1].1999999999999

A conversion of that to decimal128 would be equivalent to computing

4953959590107545m / 4503599627370496m

Which would produce

1.099999999999999866773237044981215

Repeating that for 1.2 produces

0x13333333333333L
5404319552844595m / 4503599627370496m
1.199999999999999955591079014993738

With this data, you can compute as well as I can what the various
permutations of 1.2 - 1.1 would produce for various combinations of
binary64 and decimal128 quantities.

But that being said, I recognize that any choice we make here is a
tradeoff.  Making binary64 contagious makes life more difficult for
those that are attempting to produce a decimal result.  Doug's "use
decimal" suggestion can mitigate that.  As well as providing a
Decimal.subtract method with provisions for roundingMode and (perhaps
someday) precision helps.

As would providing a Decimal.quantize
<http://speleotrove.com/decimal/dnnumb.html#numbquan>

js> Decimal.quantize(1.199999999999999955591079014993738m, 0.01)
1.20

> Finally, I'd like to take a poll: Other than people working on decimal
> at IBM and people on the EcmaScript committee, is there anyone on this
> list who thinks that decimal adds significant value to EcmaScript? If
> so, please speak up. Thanks.
>
> --
>    Cheers,
>    --MarkM

- Sam Ruby


More information about the Es-discuss mailing list