# Thoughts on IEEE P754

Sam Ruby rubys at intertwingly.net
Fri Aug 22 15:49:32 PDT 2008

```On Fri, Aug 22, 2008 at 5:46 PM, Mark S. Miller <erights at google.com> wrote:
> On Fri, Aug 22, 2008 at 11:39 AM, Waldemar Horwat <waldemar at google.com> wrote:
>> Sam Ruby wrote:
>>> When dealing with currency, having 0.05 + 0.05 produce 0.10 is a feature.
>>
>> Contrary to some beliefs, the different cohort members in IEEE P754 are *not* a valid indicator of precision.  To prove this, assume that the different cohort members are an indicator of precision.  Then evaluate:
>>
>>  3.00 + 1.00000
>>
>> IEEE P754 returns 4.00000, while the result is actually precise to only two digits after the decimal point.  Hence a contradiction.  QED.
>
> This example makes me realize how utterly confused and ignorant I am
> about the rationale for decimal.

I do know that if I have \$5 and add a dime, I expect to have \$5.10, and not \$5.

I do believe that the folks who worked on IEEE P754 worked through a
lot of scenarios, and can provide exquisite detail for the rationale
for all of these operations.

Yes, I understand that many of these results seem foreign to those of
us (myself included) who have been poisoned by years of only having
binary64 floating point available to us.  I will say that when faced
with artificial examples, my intuition tends to fail me until I find
corresponding examples using currency.

> In particular, on one of the ES3.1 calls, Sam explained why he thought
> mixed mode arithmetic should convert to binary -- IIUC, in order to
> faithfully represent our ignorance of the precision that should be
> associated with the result. From what I thought I'd understood, I
> would have expected the above example to produce 4.00. However, the
> above example makes clear that decimal by itself, even when non-mixed,
> does not attempt to accurately (or conservatively) represent one's
> ignorance of precision.

When talking about decimal128, I believe that 5 is exact, and 0.10 is
exact, and the result of adding the two should also be exact.

When talking about binary64, clearly 5 is also exact, but 0.10 can not
be represented exactly.

> Sam, did I get your rationale wrong? Now that we've seen these
> examples from Waldemar, could you try explaining again why mixed
> should result in binary?

I find it easier to talk about real examples than abstractions.  I've
done the following quickly, so forgive me if I get some detail wrong.

A binary floating point number has 52 bits of fraction, and by
means that 1.1 is stored as (for brevity, I'll use hex)

[1].1999999999999

A conversion of that to decimal128 would be equivalent to computing

4953959590107545m / 4503599627370496m

Which would produce

1.099999999999999866773237044981215

Repeating that for 1.2 produces

0x13333333333333L
5404319552844595m / 4503599627370496m
1.199999999999999955591079014993738

With this data, you can compute as well as I can what the various
permutations of 1.2 - 1.1 would produce for various combinations of
binary64 and decimal128 quantities.

But that being said, I recognize that any choice we make here is a
tradeoff.  Making binary64 contagious makes life more difficult for
those that are attempting to produce a decimal result.  Doug's "use
decimal" suggestion can mitigate that.  As well as providing a
Decimal.subtract method with provisions for roundingMode and (perhaps
someday) precision helps.

As would providing a Decimal.quantize
<http://speleotrove.com/decimal/dnnumb.html#numbquan>

js> Decimal.quantize(1.199999999999999955591079014993738m, 0.01)
1.20

> Finally, I'd like to take a poll: Other than people working on decimal
> at IBM and people on the EcmaScript committee, is there anyone on this
> list who thinks that decimal adds significant value to EcmaScript? If
> so, please speak up. Thanks.
>
> --
>    Cheers,
>    --MarkM

- Sam Ruby
```