ES Decimal status

Mike Cowlishaw MFC at uk.ibm.com
Tue Sep 23 04:32:37 PDT 2008


> There is still a blocking issue that's been discussed a lot but left
> off the issues here:
> 
> - Treatment of cohorts in the default conversion of decimal to 
> string.  Despite objections, I still get the following results:
> 
> 1/3m - 1/3m produces 0e-34
> 
> 1e12m - 1e12m produces 0e+12
> 
> 1/.1m produces 1e+1

I'm still not clear what your objections are -- we can discuss on 
Thursday, but the intent here is that the toString conversion be 
reversible (i.e., from the string representation one can recreate the 
decimal128 from which it was derived).  So I think you are really 
complaining about the rule for division (which is 'the best yet' that 
anyone has come up with, as it matches the rule for multiplication) and 
the rule for subtraction (which preserves the smaller exponent).  Hence 
1.23-1.22 -> 0.01 and 1.23-1.23 -> 0.00.   The exponent is not 
data-dependent unless there is rounding.

Mike






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU








More information about the Es-discuss mailing list