Why decimal?
Christian Plesner Hansen
christian.plesner.hansen at gmail.com
Wed Jun 24 05:33:38 PDT 2009
> School math is accurate, there are no approximations at all. Neither
> binary nor decimal are accurate, both will violate people's
> expectations of accuracy. If you do (1.0 / 3.0) * 3.0 neither binary
> nor decimal give you 1.0.
D'oh! I just tried this and it actually gives 1.0 using both 32-bit
and 64-bit binary arithmetic. But it's easy to construct examples
where none of the systems are accurate, for instance
x = 2/3
print((0.5 + x) - x);
This gives, depending on which arithmetic system you use:
128-bit decimal: 0.5000000000000000000000000000000003
64-bit binary: 0.4999999999999999
64-bit decimal: 0.5000000000000003
32-bit binary: 0.50000006
Using school math you would expect that adding and then subtracting
the same number would get you back where you started. Decimal is a
counter-intuitive as binary in that respect.
More information about the es-discuss
mailing list