Two interoperable implementations rule

[Maciej Stachowiak]

On Jul 11, 2008, at 3:49 PM, Jeff Dyer wrote:

>
> On 7/11/08 3:01 PM, Maciej Stachowiak wrote:
>
>> 2) How is interoperability to be demonstrated? Do we accept good- 
>> faith
>> claims of support, or do we need a test suite?
>
> I'd say that good faith is good enough. It's easy enough for us to  
> check
> each other's work. And the blogosphere will not be kind to liars.

I'm less concerned about cheating than about honest mistakes, which  
may nonetheless affect interoperability, Web compatibility, or  
practical implementability of the spec.

For the WebKit project, we always make our best effort to correctly  
implement Web standards, and even make our own test cases as we go.  
However, once an independently developed test suite appears it always  
finds mistakes in our implementation. I think we are not unusual in  
this regard.

> One more detail:
>
> 3) What constitutes a qualifying implementation? Does Rhino, EJScript
> (mbedthis.com), or Flash Player qualify? Or must it be one of the four
> leading browsers?

That is a good point to raise. I think limiting to browser-hosted  
implementations might be too extreme. On the other hand, if a spec  
qualifies based solely on non-browser-hosted implementations, then we  
have not done much to verify that the standard is compatible with the  
real-world Web. I think a reasonable middle ground would be to require  
at least one of the implementations to be browser-hosted. For these  
purposes, I would count an implementation that works as a browser  
extension replacing the scripting engine so long as it actually gets  
wide testing, so for example ScreamingMonkey could qualify.

It should also be required that the two implementations are  
independent (so a single vendor presenting two implementations would  
not qualify). This may be tricky to define, since many possible  
candidate implementations are open source and developed  
collaboratively by community contributors and overlapping sets of  
vendors. For example, would Rhino and SpiderMonkey count as  
sufficiently independent implementations?

>
>> Given the nature of programming languages and the high stakes of Web
>> standards, I would personally prefer whole-spec granularity  
>> (different
>> implementations having different mixes of features does not prove  
>> real
>> interoperability), and a test suite rather than just bare claims of
>> support.
>
> Again, it will be hard to get away with cheating. But, on the other  
> hand an
> unofficial test suite (such as Spidermonkey's) would make it easier  
> for
> implementors to be honest.

Again, I am less worried about cheating than mistakes. If we had a  
semi-official test suite, it would not be normative, only the spec is  
normative. It would only be a tool for verifying interoperability has  
been achieved to a reasonable degree. The concern is less about  
deliberate deception than about having at least the minimal evidence  
needed to make a fact-based claim.

Regards,
Maciej