Announcing TC39 ECMAScript Test Suite Project - Test262
Allen.Wirfs-Brock at microsoft.com
Tue Jun 22 17:29:51 PDT 2010
(I'm going to try to shift this thread to the test262-discuss list, by bccing: es-discuss.)
I think we all agree that a web based test runner is part of what we want to have. It seems inevitable that some sort of pass % or similar score has to display even though it's not clear whether such a score is very meaningful for this style of test suite. You can get big fluctuation because a single bugs can cause many individual tests to fail while in other cases a really significant bugs might only show up in a small number.
If anybody has any ideas for a meaningful metric that doesn't have those sorts of sensitivities, let's hear it.
> -----Original Message-----
> From: Christian Plesner Hansen [mailto:plesner at quenta.org]
> Sent: Tuesday, June 22, 2010 4:00 PM
> To: Sam Tobin-Hochstadt
> Cc: Allen Wirfs-Brock; test262-discuss at mozilla.org; Jonathan Chetwynd; es-
> discuss at mozilla.org
> Subject: Re: Announcing TC39 ECMAScript Test Suite Project - Test262
> Yes, there is an acidic element to this project. There will eventually be a website
> where anyone can test their browser's conformance against the test262 suite
> like they can today against the acid tests.
> -- Christian
> On Tue, Jun 22, 2010 at 3:47 AM, Sam Tobin-Hochstadt <samth at ccs.neu.edu>
> > On Mon, Jun 21, 2010 at 1:42 PM, Allen Wirfs-Brock
> > <Allen.Wirfs-Brock at microsoft.com> wrote:
> >> The intent is a suite that tests an implementation against the actual
> requirement of the Ecma-262 standard. It will be based upon and similar to what
> you can already see in the Sputnik and ES5conform suites. Those tests generally
> identify a specific requirement of a specific section of the spec. and then try to
> prove that the requirement is/isn't met by an implementation.
> >> Give that the standard specification itself is not particularly understandable
> to the general public, it probably isn't reasonable to expect that all the tests of
> the specification would be understandable to the same public.
> >> However, reasonable understandability is a good goal. What are some of the
> characteristics of the W3C suites that make them "acidic"?
> > I believe that the various "Acid" tests for rendering are what's being
> > referred to here . I think the relevant characteristc is that
> > anyone can run them in their browser, and they display a visual
> > rendering of how "good" a job the browser does, making them into
> > easily-distinguishable markers of conformance to whatever is tested.
> > It's not clear to me whether this would be good for this test suite.
> >  http://en.wikipedia.org/wiki/Acid3
> > --
> > sam th
> > samth at ccs.neu.edu
> > _______________________________________________
> > test262-discuss mailing list
> > test262-discuss at mozilla.org
> > https://mail.mozilla.org/listinfo/test262-discuss
More information about the es-discuss