Thoughts on IEEE P754

Waldemar Horwat waldemar at google.com
Fri Aug 22 12:55:13 PDT 2008


Sam Ruby wrote:
> On Fri, Aug 22, 2008 at 2:28 PM, Waldemar Horwat <waldemar at google.com> wrote:
>> Sam Ruby wrote:
>>> Waldemar Horwat wrote:
>>>> Some tidbits about our embedding of decimal:
>>>>
>>>> - Contagion should be towards decimal if decimal and binary are mixed
>>>> as operands.  5.3m + 1 should be 6.3m, not 6.3.  If we use 128-bit
>>>> decimal, this also makes the behavior of heterogeneous comparisons
>>>> (binary compared to decimal) sensible.
>>> What should 5.3m + 1.0000000000000001 produce?
>>>
>>> I also don't understand the heterogeneous comparisons comment.  What
>>> should 1.0000000000000001 == 1.0000000000000001m produce?
>> Depends.  There are many decimal formats.  Which decimal format and representation are you specifying?
> 
> Decimal128.  But the key to this question is the fact that the
> binary64 floating point constant is indistinguishable from the value
> 1.

If you're using Decimal128, then 1.0000000000000001 == 1.0000000000000001m is false because the double on the left evaluates to 1.  If you're using Decimal64, then 1.0000000000000001 == 1.0000000000000001m is true because both sides evaluate to their respective versions of 1.


Here's a question for you:

What should the full result of converting 1.1 to a Decimal be in ECMAScript?  Describe the precise answer and justify your choice of answer.  Hint:  This would behave remarkably differently depending on whether we settle on Decimal64 or Decimal128.

    Waldemar


More information about the Es-discuss mailing list