Consistent decimal semantics
waldemar at google.com
Mon Aug 25 16:41:00 PDT 2008
We're going round in circles on a few issues surrounding Decimal. Some of these have clear resolutions; others have a little wiggle room. Here's my summary:
- Should decimal values behave as objects (pure library implementation) or as primitives?
If they behave as objects, then we'd get into situations such as 3m != 3m in some cases and 3m == 3m in other cases. Also, -0m != 0m would be necessary. This is clearly unworkable.
- Should == and === distinguish cohort members?
The only reasonable answer is no, and is consistent with existing treatment of -0 and +0. Otherwise you'd break integer arithmetic: 1e12m - 1e12m === 0m would be false.
- What should cross-type == do on 5 == 5m?
Here there are a couple sensible choices: It could treat all decimal values as different from all Number values, or it could convert both values to a common type (decimal because it's wider) and compare.
My inclination would be the latter, which would make 5 == 5m true. Note that, unless we choose option 2b below, 5.1 == 5.1m would be false because 5.1 is equal to 5.099999999999999644728632119949907m.
- What should cross-type === do on 5 === 5m?
These objects are of different types, so it should return false.
- How should mixed type arithmetic work in general?
There are a few consistent design alternatives:
1. Always disallow it. For consistency you'd have to disallow == between Number and decimal as well.
2. Allow it, converting to the wider type (Decimal128). There are a couple design choices when doing the conversion:
2a. Convert per the IEEE P754 spec: 5.1 turns into 5.099999999999999644728632119949907m. This is how most programming languages operate (C++, Java, Lisp, etc.) when converting among the built-in floating point values (float -> double etc.).
2b. Convert the Number to a string and then to a Decimal: 5.1 turns into 5.1m. As a special case, -0 would turn into -0m. This might work, but I haven't thought through the implications of this one.
3. Allow it, converting to the narrower type (double). This would break transitivity: 5.00000000000000001m != 5m, but they're both == to 5, so it's a non-starter.
- Should trailing zeroes be printed when doing toString on decimal values?
No. If you print while distinguishing cohort members then you'll be required to print -0m as "-0" (which we don't do for Numbers), 1m/.1m as "1e1", and will get nasty results when using decimal numbers as array indices (barring introducing yet further complications into the language). Furthermore, were you to do an implementation with a "big red switch" that turns all numbers into decimal, everything would break because toString would print excess digits when printing even simple, unrounded values such as 1.00.
- Should + concatenate decimal numbers?
- How many double NaNs should we have?
Edition 3 says there's only one, but IEEE P754's totalOrder now distinguishes between NaN and -NaN as well as different NaNs depending on how they got created. Depending on the implementation, this is a potential compatibility problem and an undesirable way for implementations to diverge.
- How many decimal NaNs should we have?
Presumably as many as we have double NaNs....
More information about the Es-discuss