c-c external tests (like External dependent tests in gecko and gaia)
Joshua Cranmer 🐧
Pidgeot18 at gmail.com
Wed Aug 13 20:09:22 UTC 2014
On 8/13/2014 2:00 PM, Kent James wrote:
> I've often wondered if it would not make sense to have testing
> available to external accounts within comm-central, as is discussed
> below in planning for Cloud Services. We could have test accounts on
> various email providers, and do automatic testing of a test suite
> against real email servers. That's how I do my testing on ExQuilla
> using xpcshell tests.
> This must have been discussed at some point in the past, and rejected.
> Does anyone now the history of this?
Actually, it would be very useful to have tests against real servers,
particularly for extremely hard-to-mock things like GSSAPI or LDAP. The
problem with testing against real email servers in particular are the
1. Uptime of accounts (not so much a problem for the largest webmail
2. Concurrency issues (two developers happen to run their tests at the
same time--how do you deal with that?)
3. Reproducibility issues. If email providers do A/B testing, you can
end up with tests that fail for some developers but not others.
4. Account limits--automated tests that run relatively frequently can
end up accidentally triggering lockouts.
If Mozilla ran some servers on its networks for our uses (e.g., an
Exchange server, Dovecot, Cyrus, etc.), and we could give out access to
these servers for tests and contributors, that would help much of the
problem. I recall Gaia had plans to switch to real servers for mail
testing; CC'ing asuth for more information about what Gaia's plans were
Thunderbird and DXR developer
Source code archæologist
More information about the tb-planning