Artifact Builds in Automation, now with green tests

Christopher Manchester chmanchester at
Fri Mar 3 21:09:16 UTC 2017

A quick update, because I've seen a few questions about this. Bug 1315032
<> has landed, meaning
if you typically develop with "--enable-artifact-builds" locally,
"--artifact" will be added to your try syntax by "mach try". This can be
undone by either passing "--no-artifact" to "mach try" or removing
"--enable-artifact-builds" from your mozconfig.

Some jobs aren't compatible with artifact builds yet. Most of these have
pending fixes or will be investigated shortly, but this has some potential
to be confusing. In particular, some Tier-2 TaskCluster builds and the
"Sym" job are failing.

We've focused on greening up Tier-1 builds and tests so far -- these jobs
should have reliable results.


On Tue, Feb 28, 2017 at 2:22 PM, Christopher Manchester <
chmanchester at> wrote:

> Yes, thank you for pointing this out -- when going over these tests many
> of them were failing locally for the same reason, and started passing when
> the relevant fixes landed. If you notice tests that fail locally and not in
> automation, please files those as bugs as well.
> Chris
> On Tue, Feb 28, 2017 at 2:07 PM, Jared Wein <jwein at> wrote:
>> This is great news! It also means that these tests will pass when run
>> locally too, right?
>> I remember spending a time trying to figure out why a test was failing
>> when I didn't touch any of the code that it interacted with. I first
>> noticed the failure when I pushed to tryserver. I reproduced it locally.
>> Even unapplying my patches it still failed. Even pulling latest it still
>> failed. Only after filing a bug did I get feedback that it was because I
>> was using an artifact build.
>> So... thank you very much! Artifact builds are the future for front-end
>> development, and make it really easy for new contributors to get on board.
>> Cheers,
>> Jared
>> On Tue, Feb 28, 2017 at 4:05 PM, Christopher Manchester <
>> chmanchester at> wrote:
>>> Tests run against artifact builds in automation (those triggered by the
>>> --artifact argument to mach try and in try syntax) have been greened up,
>>> and as of late last week should be able to run most tests in our
>>> automation. Excluded for now are Cpp unit tests, GTests, and Jittests;
>>> triggering these against artifact builds will fail.
>>> Some improvements to the mach try command are on the way to make these
>>> easier to trigger, and to allow turning these on by default for try pushes
>>> if --enable-artifact-builds is found locally -- this is bug 1315032
>>> <>.
>>> Most issues with tests were related to test support programs that
>>> weren't downloaded and extracted accurately by artifact builds. It's
>>> possible this will come up with future tests. Please file issues found
>>> against bug 1278699
>>> <>.
>>> Chris
>>> _______________________________________________
>>> firefox-dev mailing list
>>> firefox-dev at
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the firefox-dev mailing list