test262-discuss Digest, Vol 4, Issue 2

Mark S. Miller erights at google.com
Mon Jan 17 14:59:30 PST 2011


On Mon, Jan 17, 2011 at 2:49 PM, Dave Fugate <dfugate at microsoft.com> wrote:

> Hi Mark, basically what's going on is the original test harness expected
> either a return value of 'true' or falsy for all IE Test Center-based test
> cases.  The problem with this is Sputnik tests don't return anything so the
> harness was altered at some point in time to accept 'true' or 'undefined' as
> success values.  This change on its own is harmless provided all of
> Microsoft's tests explicitly return true or false - as you've discovered
> they do not.  If you're interested, Chrome 8.0 also had the following false
> positives:
> - 11.1.5_4-4-b-1
> - 11.1.5_4-4-b-2
> - 11.1.5_4-4-c-1
> - 11.1.5_4-4-c-2
> - 11.1.5_4-4-d-1
> - 11.1.5_4-4-d-2
> - 11.1.5_4-4-d-3
> - 11.1.5_4-4-d-4
> - 11.4.1-4.a-5
> - 11.4.1-4.a-7
> - 15.2.3.3-4-178
> - 15.2.3.3-4-179
> - 15.2.3.3-4-180
> - 15.2.3.3-4-188
> - 15.4.4.14-1-1
> - 15.4.4.14-1-2
> - 15.5.4.20-1-1
> - 15.5.4.20-1-2
> - 15.5.4.20-4-10
> - 15.5.4.20-4-18
> - 15.5.4.20-4-34
> - 15.10.6
> - 15.10.7.1-1
> - 15.10.7.2-1
> - 15.10.7.3-1
> - 15.10.7.4-1
> - 15.10.7.5-1
>
> I just fixed the harness to differentiate between expected Sputnik/Test
> Center return values based on the filename (i.e., all Sputnik test cases
> begin with "S" where Test Center tests do not), and will upload this to the
> live website.
>

This seems like the wrong fix. Why not fix all tests to return either true
or false?



>
> My best,
>
> Dave
>
> -----Original Message-----
> From: Dave Fugate
> Sent: Monday, January 17, 2011 12:20 PM
> To: test262-discuss at mozilla.org
> Subject: RE: test262-discuss Digest, Vol 4, Issue 2
>
> Thanks for the report Mark!  Looks like something is up with the test
> harness here (I see the same issue under Chrome 8.0.552.237), and will
> investigate.
>
> My best,
>
> Dave
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 17 Jan 2011 01:41:37 -0800
> From: Mark Miller <erights at gmail.com>
> To: test262-discuss at mozilla.org
> Subject: test262 shows a pass for a failing testcase
> Message-ID:
>        <AANLkTi=rA+roQpYGDhTD=1McpkNgVqXNtfhEaUpc+Xf8 at mail.gmail.com<1McpkNgVqXNtfhEaUpc%2BXf8 at mail.gmail.com>
> >
> Content-Type: text/plain; charset=ISO-8859-1
>
> After running test262 on Chrome 10.0.634.0 dev, the results page shows
>
> > 15.10.7.2-1    RegExp.prototype.global is of type Boolean   [source]
> pass
>
> Clicking on the source gives
>
>
> > Test?15.10.7.2-1
> >
> > Description
> >
> > RegExp.prototype.global is of type Boolean
> >
> >
> > Testcase
> >
> > function testcase() {
> >   if((typeof(RegExp.prototype.global)) === 'boolean')
> >     return true;
> >  }
> >
> > Path
> >
> > TestCases/chapter15/15.10/15.10.7/15.10.7.2/15.10.7.2-1.js
> >
>
> But on Chrome 10.0.634.0 dev, RegExp.prototype is an object, not a RegExp,
> and it has no 'global' property. The above typeof returns the string
> 'undefined',so  true is not returned, the function falls off the end
> returning undefined which is falsy.
>
> Why does this show as a passing test?
>
> --
>
> ? ? Cheers,
> ? ? --MarkM
>
>
> ------------------------------
>
> _______________________________________________
> test262-discuss mailing list
> test262-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/test262-discuss
>
>
> End of test262-discuss Digest, Vol 4, Issue 2
> *********************************************
>
> _______________________________________________
> test262-discuss mailing list
> test262-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/test262-discuss
>



-- 
    Cheers,
    --MarkM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/test262-discuss/attachments/20110117/559ab5c9/attachment.html>


More information about the test262-discuss mailing list