Thoughts on IEEE P754
Waldemar Horwat
waldemar at google.com
Fri Aug 22 12:55:13 PDT 2008
Sam Ruby wrote:
> On Fri, Aug 22, 2008 at 2:28 PM, Waldemar Horwat <waldemar at google.com> wrote:
>> Sam Ruby wrote:
>>> Waldemar Horwat wrote:
>>>> Some tidbits about our embedding of decimal:
>>>>
>>>> - Contagion should be towards decimal if decimal and binary are mixed
>>>> as operands. 5.3m + 1 should be 6.3m, not 6.3. If we use 128-bit
>>>> decimal, this also makes the behavior of heterogeneous comparisons
>>>> (binary compared to decimal) sensible.
>>> What should 5.3m + 1.0000000000000001 produce?
>>>
>>> I also don't understand the heterogeneous comparisons comment. What
>>> should 1.0000000000000001 == 1.0000000000000001m produce?
>> Depends. There are many decimal formats. Which decimal format and representation are you specifying?
>
> Decimal128. But the key to this question is the fact that the
> binary64 floating point constant is indistinguishable from the value
> 1.
If you're using Decimal128, then 1.0000000000000001 == 1.0000000000000001m is false because the double on the left evaluates to 1. If you're using Decimal64, then 1.0000000000000001 == 1.0000000000000001m is true because both sides evaluate to their respective versions of 1.
Here's a question for you:
What should the full result of converting 1.1 to a Decimal be in ECMAScript? Describe the precise answer and justify your choice of answer. Hint: This would behave remarkably differently depending on whether we settle on Decimal64 or Decimal128.
Waldemar
More information about the Es-discuss
mailing list