Consistent decimal semantics

Waldemar Horwat waldemar at google.com
Mon Aug 25 16:41:00 PDT 2008


We're going round in circles on a few issues surrounding Decimal.  Some of these have clear resolutions; others have a little wiggle room.  Here's my summary:

- Should decimal values behave as objects (pure library implementation) or as primitives?

If they behave as objects, then we'd get into situations such as 3m != 3m in some cases and 3m == 3m in other cases.  Also, -0m != 0m would be necessary.  This is clearly unworkable.


- Should == and === distinguish cohort members?

The only reasonable answer is no, and is consistent with existing treatment of -0 and +0.  Otherwise you'd break integer arithmetic:  1e12m - 1e12m === 0m would be false.


- What should cross-type == do on 5 == 5m?

Here there are a couple sensible choices:  It could treat all decimal values as different from all Number values, or it could convert both values to a common type (decimal because it's wider) and compare.

My inclination would be the latter, which would make 5 == 5m true.  Note that, unless we choose option 2b below, 5.1 == 5.1m would be false because 5.1 is equal to 5.099999999999999644728632119949907m.


- What should cross-type === do on 5 === 5m?

These objects are of different types, so it should return false.


- How should mixed type arithmetic work in general?

There are a few consistent design alternatives:

1.  Always disallow it.  For consistency you'd have to disallow == between Number and decimal as well.
2.  Allow it, converting to the wider type (Decimal128).  There are a couple design choices when doing the conversion:
2a.  Convert per the IEEE P754 spec:  5.1 turns into 5.099999999999999644728632119949907m.  This is how most programming languages operate (C++, Java, Lisp, etc.) when converting among the built-in floating point values (float -> double etc.).
2b.  Convert the Number to a string and then to a Decimal:  5.1 turns into 5.1m.  As a special case, -0 would turn into -0m.  This might work, but I haven't thought through the implications of this one.
3.  Allow it, converting to the narrower type (double).  This would break transitivity:  5.00000000000000001m != 5m, but they're both == to 5, so it's a non-starter.


- Should trailing zeroes be printed when doing toString on decimal values?

No.  If you print while distinguishing cohort members then you'll be required to print -0m as "-0" (which we don't do for Numbers), 1m/.1m as "1e1", and will get nasty results when using decimal numbers as array indices (barring introducing yet further complications into the language).  Furthermore, were you to do an implementation with a "big red switch" that turns all numbers into decimal, everything would break because toString would print excess digits when printing even simple, unrounded values such as 1.00.


- Should + concatenate decimal numbers?

No.


- How many double NaNs should we have?

Edition 3 says there's only one, but IEEE P754's totalOrder now distinguishes between NaN and -NaN as well as different NaNs depending on how they got created.  Depending on the implementation, this is a potential compatibility problem and an undesirable way for implementations to diverge.


- How many decimal NaNs should we have?

Presumably as many as we have double NaNs....


    Waldemar


More information about the Es-discuss mailing list