<div dir="ltr"><div>Note that we also have bug 920191 on file for making JS exceptions fatal in the harness. One of the big blockers to that has always been cleaning up the existing set of problems, but maybe it would be helpful for some of these issues if/when we could drive that into happening.</div><div><br></div><div>-Ryan<br></div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Oct 18, 2017 at 12:20 PM, J. Ryan Stinnett <span dir="ltr"><<a href="mailto:jryans@gmail.com" target="_blank">jryans@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>This may not be what you want here, but just so you are aware of the option...</div><div><br></div><div><span style="font-family:arial,helvetica,sans-serif">You can us</span>e `Services.console.<wbr>registerListener`[1] in a test harness, etc. to hear messages logged to the console. The harness could count messages of a certain type (like script errors) and fail the test if there's an unexpected count (similar to assertion handling in some harnesses).</div><div><br></div><div>Some test harnesses already register console listeners, but many of them just echo the messages to the test log without examining them further.</div><div><br></div><div>For JS errors logged to the console, you can QI them to nsIScriptError and examine more details like line, column, stack, etc.</div><div><br></div><div>Anyway, might not cover all the cases envisioned here, but it's one tool available to us.<br></div><div><br></div><div>[1]: <a href="http://searchfox.org/mozilla-central/search?q=console.registerListener&case=true®exp=false&path=" target="_blank">http://searchfox.org/mozilla-<wbr>central/search?q=console.<wbr>registerListener&case=true&<wbr>regexp=false&path=</a></div><span class="HOEnZb"><font color="#888888"><div><br></div><div>- Ryan<br></div></font></span></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Wed, Oct 18, 2017 at 6:51 AM, Mark Banner <span dir="ltr"><<a href="mailto:mbanner@mozilla.com" target="_blank">mbanner@mozilla.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<div text="#000000" bgcolor="#FFFFFF">
Looping in firefox-dev as well, as I thin this is an important
discussion.<span><br>
<br>
On 18/10/2017 09:28, David Teller wrote:<br>
<blockquote type="cite">
<pre> Hi everyone,
Yesterday, Nightly was broken on Linux and MacOS because of a typo in
JS code [1]. If I understand correctly, this triggered the usual
"undefined is not a function", which was
1/ uncaught during testing, as these things often are;</pre>
</blockquote></span>
Part of the reason it was uncaught, is that there's no automated
test coverage for that bit of code. It is a migration from one
version to the other, but unless there is an explicit test (and I
don't see one) that line won't be hit.<br>
<br>
This seems to be a common theme with various migration routes in the
code base, when I've asked about ensuring more automated testing
there before I feel like I've got shrugs.<br>
<br>
Admittedly, a lot of it is simple code, and should be correct, but
as we've seen, mistakes can be made. Should we now be changing our
policy?<br>
<br>
At a minimum, I wonder if we could write some sort of simple test
for CustomizableUI that would go through most routes (with minimal
updates for each change).<span><br>
<br>
<blockquote type="cite">
<pre>2/ basically impossible to diagnose in the wild, because there was no
error message of any kind.</pre>
</blockquote></span>
I did an experiment, and the only way I got an error out was to have
"javascript.options.strict" on and
"browser.dom.window.dump.enabl<wbr>ed". Unfortunately the breakage was
such that the browser console couldn't be opened, so you'd just
having strict on wasn't enough.<span><br>
<blockquote type="cite">
<pre>I remember that we had bugs of this kind lurking for years in our
codebase, in code that was triggered daily and that everybody believed
to be tested.
I'd like to think that there is a better way to handle these bugs,
without waiting for them to explode in our user's face. Opening this
thread to see if we can find a way to somehow "solve" these bugs, either
by making them impossible, or by making them much easier to solve.</pre>
</blockquote></span>
ESLint has caught some bugs - mainly undefined and unused related
issues, and is spread through most of the production javascript
code. Unfortunately it isn't able to catch this class of error. For
that, we'd need something like Flow. Last time I looked at it, it
didn't feel like a good fit for us, although I didn't go too deep,
and I think there may have been other people that were looking at
it.<span><br>
<blockquote type="cite">
<pre>I have one proposal. Could we change the behavior of the JS VM as follows?
- The changes affect only Nightly.
- The changes affect only mozilla chrome code (including system add-ons
but not user add-ons or test code).
- Any programmer error (e.g. SyntaxError) causes a crash a crash that
displays (and attaches to the CrashReporter) both the JS stack in and
the native stack.
- Any SyntaxError is a programmer error.
- Any TypeError is a programmer error.
I expect that this will find a number of lurking errorsy, so we may want
to migrate code progressively, using a directive, say "use strict
moz-platform" and static analysis to help encourage using this directive.
</pre>
</blockquote></span>
It would definitely be interesting to fail tests on some of the
strict failures. I was surprised when I recently turned on the pref
again to see how many warnings there were.<br>
<br>
Having looked through a few of those, I've just found at least <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1409691" target="_blank">one
issue</a>, though non-critical, and I'm thinking we want to get
the <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1409693" target="_blank">no-unused-expressions</a>
rule turned on for ESLint as a result.<span class="m_-4207729325568285318HOEnZb"><font color="#888888"><br>
<br>
Mark<br>
</font></span></div>
<br></div></div><span class="">______________________________<wbr>_________________<br>
firefox-dev mailing list<br>
<a href="mailto:firefox-dev@mozilla.org" target="_blank">firefox-dev@mozilla.org</a><br>
<a href="https://mail.mozilla.org/listinfo/firefox-dev" rel="noreferrer" target="_blank">https://mail.mozilla.org/listi<wbr>nfo/firefox-dev</a><br>
<br></span></blockquote></div><br></div>
<br>______________________________<wbr>_________________<br>
firefox-dev mailing list<br>
<a href="mailto:firefox-dev@mozilla.org">firefox-dev@mozilla.org</a><br>
<a href="https://mail.mozilla.org/listinfo/firefox-dev" rel="noreferrer" target="_blank">https://mail.mozilla.org/<wbr>listinfo/firefox-dev</a><br>
<br></blockquote></div><br></div>