<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">I would suggest:</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">1. integration tests with mail servers.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Not the unit tests that we have, which
test specific code parts and do not depend on the internet, but
tests that explicitly include contacting third party Internet
provider servers. If Thunderbird no longer works with Yahoo due to
changes at Yahoo servers, we want to know about it immediately.
This did happen, and we didn't notice for months. Instead of a
human clicking through things, account setup, login and mail fetch
could easily be automated. We just need a dedicated test suite
with Internet access and test accounts for some providers.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">I would suggest, based on the most
popular ISPs:</div>
<div class="moz-cite-prefix">
<ul>
<li>Gmail freemail account</li>
<li>Gmail business domain account</li>
<li>Yahoo</li>
<li>T-Online</li>
<li>GMX</li>
<li>Web.de</li>
<li>outlook.com freemail account</li>
<li>Office365 with password</li>
<li>Office365 with MFA</li>
<li>Exchange server (on-premise) (from a friendly admin)<br>
</li>
<li>Dovecot (donated test account from a friendly admin)</li>
<li>Cyrus (ditto)</li>
</ul>
</div>
<div class="moz-cite-prefix">The test just does:</div>
<div class="moz-cite-prefix">
<ul>
<li>Sets up new account<br>
</li>
<li>Logs in</li>
<li>Fetches mail</li>
<li>Maybe sends an email (to one of the other test accounts)<br>
</li>
</ul>
<p>Then, the same with an existing pre-configured account
(hardcoded prefs.js), to see what happens for existing users.</p>
</div>
<p><br>
</p>
<p>2. Regular tests</p>
<p>by a QA person<br>
</p>
<p>Dimensions of the test matrix:</p>
<ol>
<li>Functions, like: setup, getting mail, moving mail to another
folder, moving mail from IMAP to local storage, starring a
contact, editing a contact, searching for a contact, writing an
email to a contact with autocomplete, sending the email as
plain+HTML, receiving the same email.</li>
<li>Operating systems: Windows, Linux, Mac</li>
</ol>
<p>Important: Not all combinations of the matrix need to be checked,
given the cross platform nature. Basic functions should be checked
on all platforms, though.</p>
<p><br>
</p>
<p>3. Investigating bug reports</p>
<p>in bugzilla, by somebody competent, who can investigate the
issues in detail. The person needs to reproduce the issue, and if
he cannot reproduce, try to find out why it fails for the reporter
and works for him. He needs to be able to look for network
traffic, etc., to investigate causes of the bug, and then assign
the bug to the correct developer. The investigation work needs to
be done by QA, not dev (too busy), and not the end user (doesn't
know how, and no time).</p>
<p><br>
</p>
<p>Ben<br>
</p>
</body>
</html>