Help needed converting mochitests to manifests

Gregory Szorc gregory.szorc at gmail.com
Mon Oct 7 20:36:32 UTC 2013


Would you like to make the tree build faster? Of course you do! Read on 
to find out how you can help.

As announced a few days ago [1], mochitest tests are now declared in 
.ini manifest files [2] instead of Makefile.in. There are a number of 
advantages to declaring tests this way. One reason is faster build times.

Mochitests in Makefile.in contribute to slower builds for 2 main reasons:

1) The file installation mechanism for make files is less efficient than 
the file installation mechanism if files are known to the moz.build 
world (via .ini manifests). When we landed the initial bulk conversion 
of mochitests to .ini files, my no-op build times decreased from ~60s to 
~50s. Not too shabby!

2) Many Makefile.in only exist to define mochitest files. Each 
Makefile.in/moz.build incurs a cost to process at build time. If we 
eliminate entire Makefile.in/moz.build files from the tree, build times 
will decrease.

Anyway, we could use your help converting mochitests to use .ini 
manifests so the build is faster.

Here's how you can help:

1) Read up on how mochitest manifests work at [2].

2a) Identify a directory still defining mochitests in Makefile.in (grep 
for MOCHITEST_FILES, MOCHITEST_CHROME_FILES, MOCHITEST_BROWSER_FILES, 
MOCHITEST_A11Y_FILES, etc). Create .ini test manifests and move the 
references from Makefile.in to the new files. Finally, add a *_MANIFESTS 
entry to a moz.build file saying whether to find the new .ini file. This 
last part is important - without it, the build system doesn't know about 
the manifest files! The relevant moz.build variables are A11Y_MANIFESTS, 
BROWSER_CHROME_MANIFESTS, MOCHITEST_CHROME_MANIFESTS, 
MOCHITEST_MANIFESTS, WEBRTC_SIGNALLING_TEST_MANIFESTS.

2b) Find directories with "empty" moz.build/Makefile.in and move the 
test manifest declarations up to a parent directory and remove the 
"useless" leaf moz.build/Makefile.in files (see "c" below).

3) File a bug blocking bug 920185 and submit a patch to perform the 
conversion. You shouldn't need review from a peer outside the module the 
tests belong to. But if you want one, hit up :gps, :ted, or :Ms2ger.

4) Everyone profits.

When you work on the patches, keep the following in mind:

a) Manifests support the "skip-if," etc directives to annotate tests. 
Please use these annotations rather than commenting out tests! See [2] 
for all available annotations.

b) Makefile.in files may be using complex or unsupported conditionals 
for filtering tests. It's entirely possible the metadata its using are 
not yet available to the test manifest files via mozinfo [3]. If you 
can't represent a filter condition in test manifests, needinfo? :ted or 
:gps and we'll work with you to get support added. Until support is 
added, you can just comment out the test in the manifest or leave 
affected tests in Makefile.in (while converting everything else).

c) You do not need a moz.build/Makefile.in in the same directory as the 
test manifest .ini file! If your moz.build/Makefile.in *only* defines 
test manifests, you should delete the "leaf" moz.build/Makefile.in files 
and move the MOCHITEST_MANIFESTS (or appropriate) moz.build entry to a 
moz.build file in a parent directory. e.g. "MOCHITEST_MANIFESTS += 
['tests/mochitests/mochitest.ini']". By reducing the total number of 
moz.build/Makefile.in files in the tree, we make builds faster. Please 
note that the already-performed automatic conversion did not attempt to 
create ideal directory structure. Many directories have nothing left in 
Makefile.in but could still use "reparenting" of the *MANIFESTS entries 
in moz.build files. We currently have 1578 moz.build files in the tree 
and 606 of them are in directories with "test" in the path. I would 
ideally like to see all 606 of those moz.build files go away. Since 
pymake overhead is largely in make file processing overhead, this would 
have drastic implications for pymake builds!

d) Theoretically we no longer need a file naming convention for 
mochitests since the manifests annotate support files from test files 
and the flavor of each test file (at least from the perspective of the 
build system). I'm not proposing any changes, just stating that there 
could be a conversation :)

If you have any questions, reply or speak up in #build or #ateam.

Gregory

[1] https://groups.google.com/forum/#!topic/mozilla.dev.platform/qXM5qMpJ1bQ
[2] 
https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/test_manifests.html
[3] 
https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/mozinfo.html



More information about the firefox-dev mailing list