<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">One topic that I wanted to start a discussion on is how we should name and organize the individual test262 test. In particular, the possibility of the test suite surviving future rewrites of the ECMAScript specification is something we
should think about now as we’re getting started. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I think the sort of general hierarchical organization down to the algorithm step level that both Sputnik and ES5Confrom use has proven useful from both a test management and a results communications perspective. However, I’m concerned
about the long term manageability of that organization. We have a good example of the problem right out of the gate. Most of the Sputnik tests are applicable to ES5 but in many cases the algorithm step level identification has changed and in some cases even
the major section numbering has changed such the test identifier doesn’t match the locator of the corresponding requirement from the specification. We could manually rename everything to match the ES5 spec. but we would probably have to do it again for “ES6”.
Plus it seems that there is value in maintaining the ability to test against different generations of the specification: ES3, ES5, “Harmony draft 1”, “ES6” etc. This could be done by copying and renaming individual tests but that creates maintenance problems
when the evitable test bugs have to be fixed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">One idea I’m been thinking about is the possibility separating the organization of the tests from the naming of individual test files. For example, tests could be given individual unique identifiers (various schemes are possible) and then
there could be separate organization “map” that follow the specification structure and maps to individual tests using their unique ids. Then we could have an organizational map for ES3, another for ES5, etc. all referencing into the same set of test. These
organization map would be used for sequencing testing and reporting results. <o:p>
</o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Does anyone have any thought on this?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Allen<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>