<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">In a freak coincidence, given the
suspected objections to a commit hook, I filed
<a class="moz-txt-link-freetext" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1348918">https://bugzilla.mozilla.org/show_bug.cgi?id=1348918</a> earlier today
to add DOM-style warnings to the browser.ini file (patch is up
already), as I happened to notice today that people are still
adding more tests.<br>
<br>
Meanwhile, many thanks to Florian, Jared and Mark who have all
moved various bits and pieces to other directories already.<br>
<br>
Gijs<br>
<br>
On 20/03/2017 20:35, Gregory Szorc wrote:<br>
</div>
<blockquote
cite="mid:CAJTgH0nSk5xb+mv=URPecX746_uWAVPHr7_iDz0gRbmmusAtUw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">The cost of a hook depends on what
level it inspects. Reading changeset metadata is the
cheapest and nearly instantaneous. Next is examining the set
of file names that changed. The slowest are hooks that need
to examine content.<br>
<br>
</div>
<div class="gmail_quote">With the movement towards Autoland,
latency of pushes becomes a lesser concern and the biggest
concern for hooks is ongoing maintenance costs. A parallel
concern is ensuring failures are caught fast - as early in
the development process as possible. We don't want
developers only finding out about bustage at landing time.<br>
</div>
<div class="gmail_quote"><br>
On Wed, Feb 22, 2017 at 6:12 AM, Ryan VanderMeulen <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:rvandermeulen@mozilla.com" target="_blank">rvandermeulen@mozilla.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>I suspect gps will have opinions about this when he
returns from PTO. Basically, I think the main concern
is that commit hooks have a performance penalty for
every push and should therefore be avoided where
possible.<span class="HOEnZb"><font color="#888888"><br>
<br>
</font></span></div>
<span class="HOEnZb"><font color="#888888">-Ryan<br>
</font></span></div>
<div class="HOEnZb">
<div class="h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Feb 22, 2017 at
9:09 AM, Jared Wein <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:jwein@mozilla.com"
target="_blank">jwein@mozilla.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>On a recent Win7 Opt bc-7 run, the
following tests took over 1s each in
/browser/base/content/test/gen<wbr>eral.
The number at the beginning is the
amount of milliseconds for the
respective test.<br>
<br>
msunu@DESKTOP-I07UH4B /c/fx<br>
$ sort dump.txt -n -r<br>
32413 took |
browser/base/content/test/gene<wbr>ral/browser_audioTabIcon.js<br>
18188 took |
browser/base/content/test/gene<wbr>ral/browser_bookmark_popup.js<br>
15910 took |
browser/base/content/test/gene<wbr>ral/browser_parsable_script.js<br>
13190 took |
browser/base/content/test/gene<wbr>ral/browser_bug1299667.js<br>
12157 took |
browser/base/content/test/gene<wbr>ral/browser_sanitizeDialog.js<br>
11302 took |
browser/base/content/test/gene<wbr>ral/browser_bug553455.js<br>
10983 took |
browser/base/content/test/gene<wbr>ral/browser_selectpopup.js<br>
10472 took |
browser/base/content/test/gene<wbr>ral/browser_syncui.js<br>
10422 took |
browser/base/content/test/gene<wbr>ral/browser_newWindowDrop.js<br>
9102 took |
browser/base/content/test/gene<wbr>ral/browser_extension_permissi<wbr>ons.js<br>
8281 took |
browser/base/content/test/gene<wbr>ral/browser_extension_update_<wbr>background.js<br>
7629 took |
browser/base/content/test/gene<wbr>ral/browser_aboutHome.js<br>
7479 took |
browser/base/content/test/gene<wbr>ral/browser_extension_update_<wbr>interactive.js<br>
6518 took |
browser/base/content/test/gene<wbr>ral/browser_contentSearchUI.js<br>
6015 took |
browser/base/content/test/gene<wbr>ral/browser_aboutAccounts.js<br>
5397 took |
browser/base/content/test/gene<wbr>ral/browser_contentAltClick.js<br>
5077 took |
browser/base/content/test/gene<wbr>ral/browser_tab_drag_drop_perw<wbr>indow.js<br>
4520 took |
browser/base/content/test/gene<wbr>ral/browser_sanitize-timespans<wbr>.js<br>
4226 took |
browser/base/content/test/gene<wbr>ral/browser_trackingUI_telemet<wbr>ry.js<br>
4209 took |
browser/base/content/test/gene<wbr>ral/browser_windowactivation.<wbr>js<br>
4094 took |
browser/base/content/test/gene<wbr>ral/browser_web_channel.js<br>
3408 took |
browser/base/content/test/gene<wbr>ral/browser_bug676619.js<br>
3148 took |
browser/base/content/test/gene<wbr>ral/browser_fullscreen-window-<wbr>open.js<br>
3059 took |
browser/base/content/test/gene<wbr>ral/browser_printpreview.js<br>
2987 took |
browser/base/content/test/gene<wbr>ral/browser_bug422590.js<br>
2924 took |
browser/base/content/test/gene<wbr>ral/browser_trackingUI_4.js<br>
2893 took |
browser/base/content/test/gene<wbr>ral/browser_bug575561.js<br>
2878 took |
browser/base/content/test/gene<wbr>ral/browser_relatedTabs.js<br>
2862 took |
browser/base/content/test/gene<wbr>ral/browser_ctrlTab.js<br>
2859 took |
browser/base/content/test/gene<wbr>ral/browser_bug590206.js<br>
2763 took |
browser/base/content/test/gene<wbr>ral/browser_refreshBlocker.js<br>
2718 took |
browser/base/content/test/gene<wbr>ral/browser_trackingUI_5.js<br>
2702 took |
browser/base/content/test/gene<wbr>ral/browser_trackingUI_1.js<br>
2659 took |
browser/base/content/test/gene<wbr>ral/browser_aboutCertError.js<br>
2630 took |
browser/base/content/test/gene<wbr>ral/browser_extension_sideload<wbr>ing.js<br>
2550 took |
browser/base/content/test/gene<wbr>ral/browser_storagePressure_<wbr>notification.js<br>
2441 took |
browser/base/content/test/gene<wbr>ral/browser_ssl_error_reports.<wbr>js<br>
2365 took |
browser/base/content/test/gene<wbr>ral/browser_temporary_permissi<wbr>ons_navigation.js<br>
2295 took |
browser/base/content/test/gene<wbr>ral/browser_bug767836_perwindo<wbr>wpb.js<br>
2279 took |
browser/base/content/test/gene<wbr>ral/browser_bug495058.js<br>
2185 took |
browser/base/content/test/gene<wbr>ral/browser_bug763468_perwindo<wbr>wpb.js<br>
2147 took |
browser/base/content/test/gene<wbr>ral/browser_beforeunload_dupli<wbr>cate_dialogs.js<br>
2138 took |
browser/base/content/test/gene<wbr>ral/browser_newwindow_focus.js<br>
2067 took |
browser/base/content/test/gene<wbr>ral/browser_overflowScroll.js<br>
2016 took |
browser/base/content/test/gene<wbr>ral/browser_getshortcutoruri.<wbr>js<br>
2014 took |
browser/base/content/test/gene<wbr>ral/browser_save_link_when_<wbr>window_navigates.js<br>
1977 took |
browser/base/content/test/gene<wbr>ral/browser_bug592338.js<br>
1976 took |
browser/base/content/test/gene<wbr>ral/browser_save_link-perwindo<wbr>wpb.js<br>
1895 took |
browser/base/content/test/gene<wbr>ral/browser_temporary_permissi<wbr>ons_tabs.js<br>
1894 took |
browser/base/content/test/gene<wbr>ral/browser_temporary_permissi<wbr>ons.js<br>
1882 took |
browser/base/content/test/gene<wbr>ral/browser_documentnavigation<wbr>.js<br>
1855 took |
browser/base/content/test/gene<wbr>ral/browser_permissions.js<br>
1818 took |
browser/base/content/test/gene<wbr>ral/browser_tab_dragdrop.js<br>
1807 took |
browser/base/content/test/gene<wbr>ral/browser_trackingUI_2.js<br>
1788 took |
browser/base/content/test/gene<wbr>ral/browser_parsable_css.js<br>
1776 took |
browser/base/content/test/gene<wbr>ral/browser_bug719271.js<br>
1752 took |
browser/base/content/test/gene<wbr>ral/browser_datachoices_notifi<wbr>cation.js<br>
1670 took |
browser/base/content/test/gene<wbr>ral/browser_newTabDrop.js<br>
1670 took |
browser/base/content/test/gene<wbr>ral/browser_misused_characters<wbr>_in_strings.js<br>
1638 took |
browser/base/content/test/gene<wbr>ral/browser_tab_dragdrop2.js<br>
1608 took |
browser/base/content/test/gene<wbr>ral/browser_e10s_switchbrowser<wbr>.js<br>
1567 took |
browser/base/content/test/gene<wbr>ral/browser_e10s_chrome_proces<wbr>s.js<br>
1444 took |
browser/base/content/test/gene<wbr>ral/browser_contextmenu_input.<wbr>js<br>
1428 took |
browser/base/content/test/gene<wbr>ral/browser_bug462673.js<br>
1357 took |
browser/base/content/test/gene<wbr>ral/browser_domFullscreen_full<wbr>screenMode.js<br>
1353 took |
browser/base/content/test/gene<wbr>ral/browser_tabkeynavigation.<wbr>js<br>
1274 took |
browser/base/content/test/gene<wbr>ral/browser_aboutHome_wrapsCor<wbr>rectly.js<br>
1232 took |
browser/base/content/test/gene<wbr>ral/browser_private_browsing_<wbr>window.js<br>
1232 took |
browser/base/content/test/gene<wbr>ral/browser_favicon_change_<wbr>not_in_document.js<br>
1213 took |
browser/base/content/test/gene<wbr>ral/browser_tab_detach_restore<wbr>.js<br>
1129 took |
browser/base/content/test/gene<wbr>ral/browser_bug735471.js<br>
1087 took |
browser/base/content/test/gene<wbr>ral/browser_zbug569342.js<br>
1081 took |
browser/base/content/test/gene<wbr>ral/browser_alltabslistener.js<br>
1056 took |
browser/base/content/test/gene<wbr>ral/browser_bug623893.js<br>
1054 took |
browser/base/content/test/gene<wbr>ral/browser_contextmenu_childp<wbr>rocess.js<br>
1042 took |
browser/base/content/test/gene<wbr>ral/browser_tabDrop.js<br>
1015 took |
browser/base/content/test/gene<wbr>ral/browser_offlineQuotaNotifi<wbr>cation.js<br>
<br>
</div>
These 79 tests take 5m30s to run, not
including time between tests which is
notable on b-c runs. In comparison, the
full test suite (238 tests in this case)
takes 6m23s to run. Treeherder reports
the full duration as 12 minutes, so you
can see the amount of overhead we've
got.<br>
<br>
</div>
I'm fine with the commit hook. The list
above should give us a good priority view
on what tests should be moved first.<br>
<br>
</div>
Cheers,<br>
</div>
Jared<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">
<div>
<div class="m_7933936065577203822h5">On
Wed, Feb 22, 2017 at 6:34 AM, Michael de
Boer <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mdeboer@mozilla.com"
target="_blank">mdeboer@mozilla.com</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div>
<div class="m_7933936065577203822h5">
<div style="word-wrap:break-word">+1
<div><span><br>
<div>
<blockquote type="cite">
<div>On 22 Feb 2017, at
11:09, Gijs Kruitbosch
<<a
moz-do-not-send="true"
href="mailto:gijskruitbosch@gmail.com"
target="_blank">gijskruitbosch@gmail.com</a>>
wrote:</div>
<br
class="m_7933936065577203822m_3637133827424155414m_-5468645579926991227Apple-interchange-newline">
<div>
<div bgcolor="#FFFFFF"
text="#000000">
<p>Hello firefox-dev,</p>
<p>A while back I tried
to start moving
(browser) mochitests
out of
browser/base/content/test/gene<wbr>ral/
to more topical
directories. To copy
my reasoning from the
original metabug:</p>
<div> <br
class="m_7933936065577203822m_3637133827424155414m_-5468645579926991227webkit-block-placeholder">
</div>
<blockquote type="cite">
</blockquote>
<blockquote type="cite">It
takes ages to run
[locally], there's a
lot of tests that can
influence each other,
it's hard to isolate a
problem if tests do
influence each other,
it's hard to run tests
relevant for the
change you're making,
it takes long on
infra, if something
breaks (cough bug
1253956 cough)
everything gets
mass-disabled... there
are a lot of reasons
to not want to have a
directory with over
300 tests in.</blockquote>
<div><br
class="m_7933936065577203822m_3637133827424155414m_-5468645579926991227webkit-block-placeholder">
</div>
<p>To add to that:
browser mochitests are
some of our slowest
tests. As far as I can
tell, the mochitest-bc
chunk on infra that
contains general/
right now *runs no
other directories at
all*. On <a
moz-do-not-send="true"
href="https://public-artifacts.taskcluster.net/KIBQPE3_QZCAnjxL8jd9zA/0/public/logs/live_backing.log"
target="_blank">linux
e10s debug</a>, that
directory takes 50
minutes (3400s) to
run.<br>
</p>
<p>There are a lot more
topical subdirs now,
and there's really no
good reason, IMO, to
keep adding random
tests to this
directory. Which does
seem to keep
happening: while the
number of tests was
266 after I moved
40-odd tests to the
urlbar/ directory 10
months ago, and more
tests were moved out
of general/ since, the
number of tests in
general/ right now is
267 (we'll run a ~90%
subset of those
depending on platform,
debug on/off and e10s
on/off).<br>
</p>
<p>I'd like to add a
commit/push hook that
makes adding more
tests impossible
unless you include
some magic words,
similar to the webidl
DOM peer review hooks.<br>
</p>
<p>Would anyone object
to this happening?</p>
</div>
</div>
</blockquote>
</div>
</span>
<div>Heck no.</div>
<br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Mike.</div>
</div>
<br>
</div>
</div>
______________________________<wbr>_________________<br>
firefox-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:firefox-dev@mozilla.org"
target="_blank">firefox-dev@mozilla.org</a><br>
<a moz-do-not-send="true"
href="https://mail.mozilla.org/listinfo/firefox-dev"
rel="noreferrer" target="_blank">https://mail.mozilla.org/listi<wbr>nfo/firefox-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
______________________________<wbr>_________________<br>
firefox-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:firefox-dev@mozilla.org"
target="_blank">firefox-dev@mozilla.org</a><br>
<a moz-do-not-send="true"
href="https://mail.mozilla.org/listinfo/firefox-dev"
rel="noreferrer" target="_blank">https://mail.mozilla.org/listi<wbr>nfo/firefox-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
firefox-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:firefox-dev@mozilla.org">firefox-dev@mozilla.org</a><br>
<a moz-do-not-send="true"
href="https://mail.mozilla.org/listinfo/firefox-dev"
rel="noreferrer" target="_blank">https://mail.mozilla.org/<wbr>listinfo/firefox-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>