External dependent tests in gecko and gaia
gps at mozilla.com
Wed Aug 13 09:39:50 PDT 2014
On 8/13/14 9:21 AM, Edwin Wong wrote:
> Hi dev-platform,
> TL;DR - Cloud Services and Quality Engineering would like to propose the creation of a directory named “external" in gecko and gaia repos for externally dependent tests.
> This enables features married to Cloud Services such as Loop, FindMyDevice, FirefoxAccounts, and Sync to have centralized tests that can be run locally or on other continuous integration systems. These tests would live in this "external” directory along side existing tests (so they live together). These will be run and sheriffed independently from the main tests. Reviews would be governed by modules and feature teams.
> More detail:
I'm glad we're going down this road!
While having a separate directory enforces the separation between tests
and jobs, this separation is partially the result of historical
necessity: some test suites looked at filenames or directory contents
for determining what to execute.
These days, tests are defined in manifest files. This means that
separate directories don't technically need to exist and the barriers
between test flavors can be less pronounced.
Have you considered annotating external-accessing tests inside manifests
and having the automation run the subset of tests it is capable of? e.g.
|mach marionette-test --no-external| or |mach test --external-only|
More information about the Dev-fxacct