# Re precision, etc.

Mike Cowlishaw MFC at uk.ibm.com
Fri Aug 22 13:07:54 PDT 2008

```Re:

> ----- Message from Waldemar Horwat <waldemar at google.com> on Fri, 22
> Aug 2008 11:39:57 -0700 -----

> Contrary to some beliefs, the different cohort members in IEEE P754
> are *not* a valid indicator of precision.  To prove this, assume
> that the different cohort members are an indicator of precision.
> Then evaluate:
>
>   3.00 + 1.00000
>
> IEEE P754 returns 4.00000, while the result is actually precise to
> only two digits after the decimal point.  Hence a contradiction.  QED.

The preservation of exponents in IEEE 754 is indeed not an indication of
'precision' in the significance arithmetic sense, and it was never
intended to be.  See:

http://speleotrove.com/decimal/decifaq2.html#precision

for some of the uses of the word 'precision' -- it is best avoided -- and
see:

http://speleotrove.com/decimal/decifaq4.html#signif

for why 'significance arithmetic' makes no sense at all, beyond single
operations and rules-of-thumb.

The rules in 754 codify current 'best' practice (the rule for division,
for example, came from the .Net and C# division operator in its decimal
class; it has the very desirable property that 2.40/2 gives 1.20, etc.).
The result of 1/.1 is a necessary consequence of that .. and a lesser evil
than not preserving the information that's available.  There might well be
a better rule for division, but that was the best proposal (and that has
been in the public drafts since 2003, so there has been plenty of time to
make alternative proposals).

On the Array indexes, I have some homework to do, obviously.  I had
thought ES arrays were indexed by integers (0 though 2^n-1) rather than by
strings (as in Rexx stems).  If an index of "0.00" is indeed different
from one of "0" than it would seem to be appropriate that the same
distinction apply if the index is a decimal -- because the whole idea of
decimal numbers is that they *are* the same as the string representation,
from the user's point of view.

Mike

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

```