Re precision, etc.

Mike Cowlishaw MFC at uk.ibm.com
Fri Aug 22 13:07:54 PDT 2008


Re:

> ----- Message from Waldemar Horwat <waldemar at google.com> on Fri, 22 
> Aug 2008 11:39:57 -0700 -----

> Contrary to some beliefs, the different cohort members in IEEE P754 
> are *not* a valid indicator of precision.  To prove this, assume 
> that the different cohort members are an indicator of precision. 
> Then evaluate:
> 
>   3.00 + 1.00000
> 
> IEEE P754 returns 4.00000, while the result is actually precise to 
> only two digits after the decimal point.  Hence a contradiction.  QED.

The preservation of exponents in IEEE 754 is indeed not an indication of 
'precision' in the significance arithmetic sense, and it was never 
intended to be.  See:

  http://speleotrove.com/decimal/decifaq2.html#precision

for some of the uses of the word 'precision' -- it is best avoided -- and 
see:

  http://speleotrove.com/decimal/decifaq4.html#signif

for why 'significance arithmetic' makes no sense at all, beyond single 
operations and rules-of-thumb.

The rules in 754 codify current 'best' practice (the rule for division, 
for example, came from the .Net and C# division operator in its decimal 
class; it has the very desirable property that 2.40/2 gives 1.20, etc.). 
The result of 1/.1 is a necessary consequence of that .. and a lesser evil 
than not preserving the information that's available.  There might well be 
a better rule for division, but that was the best proposal (and that has 
been in the public drafts since 2003, so there has been plenty of time to 
make alternative proposals).

On the Array indexes, I have some homework to do, obviously.  I had 
thought ES arrays were indexed by integers (0 though 2^n-1) rather than by 
strings (as in Rexx stems).  If an index of "0.00" is indeed different 
from one of "0" than it would seem to be appropriate that the same 
distinction apply if the index is a decimal -- because the whole idea of 
decimal numbers is that they *are* the same as the string representation, 
from the user's point of view.

Mike





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU








More information about the Es-discuss mailing list