<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">IME panels/popups/context-menus are
flaky generally, and especially on Linux. I don't know how to
meaningfully lint for this.<br>
<br>
The other thing is various incantations that try to do variations
on the "load x in a tab" theme. It turns out that it's not trivial
to not race with an expectant page load, especially if you don't
have full control over triggering it (ie the test is not the
callsite of loadURI or equivalent) and they involve new tabs, and
e10s is on as well. Worse if you also need to avoid hitting the
network (see e.g. waitForDocLoadAndStopIt for a demonstration of
this).<br>
<br>
Finally, the most common description of orange I see is some
variation of "test timed out". It would be helpful if we had more
fluid rules about test durations, so that mostly-unrelated patches
don't "suddenly" cause tests to cross the magical 45/90 second
barrier (more often) and then get backed out.<br>
<br>
~ Gijs<br>
<br>
On 06/07/2017 19:47, Andrew Halberstadt wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAAJAz+6q3JWQFAjdAOCba-QoKC7Tn69JZTWjvojBS0rrkC3NkA@mail.gmail.com">
<div dir="ltr">
<div>I'm looking into creating custom eslint rules that aim to
avoid common causes of oranges in tests. We have an <a
href="https://developer.mozilla.org/en-US/docs/Mozilla/QA/Avoiding_intermittent_oranges"
target="_blank" moz-do-not-send="true">MDN page</a>
containing some of these already. Some of those patterns might
be pretty hard to catch with a linter, but others look
do-able. So far, I'm thinking of adding rules for:</div>
<div>
<ul>
<li>No arbitrary setTimeouts</li>
<li>No synthesizeKey without waiting for focus</li>
<li>No deleting original tab</li>
<li>???<br>
</li>
</ul>
<div>These rules would only be configured to run on test
files. I have a few questions:</div>
<div>
<ol>
<li>Would this cause too much of a burden for legitimate
uses of those patterns?</li>
<li>Do any of the other patterns on that page look
feasible to implement as linters?</li>
<li>Are there additional things not listed on that page
that we could lint for?</li>
</ol>
I'd love to find volunteer(s) who are more familiar with
writing mochitests than myself (and who also care about this
issue) to help brainstorm and provide more in-depth
feedback. I want to strike a balance between preventing
oranges and not over-burdening test authors, but I'm not
sure what that balance looks like.</div>
<div><br>
</div>
<div>Thanks in advance for your input,</div>
<div>Andrew<br>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
firefox-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:firefox-dev@mozilla.org">firefox-dev@mozilla.org</a>
<a class="moz-txt-link-freetext" href="https://mail.mozilla.org/listinfo/firefox-dev">https://mail.mozilla.org/listinfo/firefox-dev</a>
</pre>
</blockquote>
<p><br>
</p>
</body>
</html>