Why decimal?

Christian Plesner Hansen christian.plesner.hansen at gmail.com
Wed Jun 24 05:33:38 PDT 2009


> School math is accurate, there are no approximations at all.  Neither
> binary nor decimal are accurate, both will violate people's
> expectations of accuracy.  If you do (1.0 / 3.0) * 3.0 neither binary
> nor decimal give you 1.0.

D'oh!  I just tried this and it actually gives 1.0 using both 32-bit
and 64-bit binary arithmetic.  But it's easy to construct examples
where none of the systems are accurate, for instance

  x = 2/3
  print((0.5 + x) - x);

This gives, depending on which arithmetic system you use:

  128-bit decimal: 0.5000000000000000000000000000000003
  64-bit binary: 0.4999999999999999
  64-bit decimal: 0.5000000000000003
  32-bit binary: 0.50000006

Using school math you would expect that adding and then subtracting
the same number would get you back where you started.  Decimal is a
counter-intuitive as binary in that respect.


More information about the es-discuss mailing list