From mozmill to mochitests (Re: Intent to de-support: traditional add-ons)

Geoff Lankow geoff at thunderbird.net
Wed Oct 2 21:04:29 UTC 2019


Three months is plenty of time to convert the remaining tests. I think 
you over-estimate how little actually needs to be done to most tests for 
them to work in Mochitest.

In most cases it involves (1) making sure the test can import the 
supporting files it needs, (2) renaming the test files to match the 
Mochitest convention and adding a manifest to each directory, and (3) 
making some small changes to the file itself. I've already done part 1 
for all the files.

GL

On 02/10/2019 22:52, ISHIKAWA,chiaki wrote:
> I think somebody ought to raise a red flag here.
>
> At last count, there were 32 test subdirectories and the total tests 
> were somewhere in 1100-1200 range.
> (A single file can contain multiple tests.)
>
> I wonder who are working on the conversion aside from Geoff.
>
> I didn't realize this 2019 timeline, BTW.
> Schedule-wise, three months remain meaning more than 10 tests (in a 
> few files) need to be converted daily.
>
> I, for one, have looked at some tests before in order to understand 
> why some tests fail due to my CPP file changes, but could not make 
> head or tail of them at all, honestly speaking.
> For example, I am trying to figure out why my local patches fail the 
> following test, but
> I could not fathom exactly what it is trying to do. Too sparse 
> documentation of mozmill, I am afraid.
>
> comm/mailnews/base/test/unit/test_quarantineFilterMove.js
>
>
> I wonder if there are enough man-power for the conversion in time.
> I am happy to be corrected if I am wrong.
>
> If mochitests is easier to understand than mozmill, that would be 
> super in the end for extending tests in general and adding new tests: 
> there are many things that don't get checked properly  in current 
> mozmill test. For example, I notice that some tests are not handling 
> error dialog correctly at all. They either look at incorrect dialog 
> and decided that the error is handled or never expected to encounter 
> extra error dialogs at all.
> In fact, there are cases where incorrect error dialog seems to be 
> processed as the expected one  and the other
> error dialog is simply ignored and timed out if I am not mistaken. You 
> can look at the mozmill test screen and see such strange error dialog 
> shown on the screen for timeout period and the test basically pauses 
> until timeout kicks in and only then whlole test proceeds again.
> The whole mozmill test takes less than two hours now on my Ryzen 1700 
> CPU and so
> you can simply run the test in the background and make the screen 
> visible, and then do some other work on PC.
> You will eventually notice that there are moments of frozen screen 
> with error dialog being shown and nothing happens. It is hard to miss.
>
> TIA
>
> Chiaki
>
>
> On 2019/10/02 18:06, Magnus Melin wrote:
>>
>> Please see bug 1571681 
>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1571681> tracking this 
>> conversion.
>>
>> In bug 1585162 
>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1585162>there is 
>> already a patch converting over all the calendar mozmill tests, 
>> probably landing soon. You can see the approach from there.
>>
>> The timeline should within this 2019.
>>
>>  -Magnus
>>
>> On 02-10-2019 12:00, ISHIKAWA,chiaki wrote:
>>> I have changed the subject line.
>>>
>>> I am focusing on a side topic.
>>>
>>> > Mozmill (used in our test infra): we're converting over to using 
>>> mochitests instead
>>>
>>> So we are moving from mozmill to mochitests.
>>>
>>> How is it planned?
>>>
>>> I would like to see a smooth transition.
>>>
>>> For example,
>>>
>>> - rewriting test one by one from mozmill to mochitests and
>>>   create a duplicate set of existing tests (functional duplicates) 
>>> so that
>>>   equivalent of all the tests are available and have been thoroughly 
>>> tested
>>>   by the time we switch from mozmill to mochitests completely.
>>>
>>> - Is there a document to describe how to rewrite mozmill tests to 
>>> mochitests to guide such conversion?
>>>
>>> (- One thing I noticed is that there *ARE* xpcshell tests for TB, 
>>> but xpcshell test framework doesn't create
>>>    visible windows during tests and that cause some local hacks to 
>>> fail since
>>>    I could not produce visible error message to the user due to lack 
>>> of visible screen
>>>    when an serious I/O error occurs. I hope mochitests won't have 
>>> such strange restriction.
>>>    Oh well, since it checks the GUI operation as well, there WILL be 
>>> screens, I suppose.)
>>>
>>> What is the planned timescale of the transition?
>>> End of 2019 is unattainable.
>>> Maybe the end of 2020?
>>>
>>> TIA
>>>
>>> Chiaki
>>>
>>>
>>>
>>> On 2019/10/01 22:02, Magnus Melin wrote:
>>>>
>>>> Since version 57, Firefox only supports add-ons through the 
>>>> WebExtensions APIs. At the time, Thunderbird decided to continue 
>>>> supporting traditional add-ons, since we hadn't yet been able to 
>>>> develop replacing APIs for add-on developers to use.
>>>>
>>>> Since then, Thunderbird has been developing WebExtensions APIs (aka 
>>>> MailExtensions), and the number of APIs available is continuously 
>>>> growing: https://thunderbird-webextensions.readthedocs.io/
>>>>
>>>> Because the toolkit support for traditional add-ons has been 
>>>> largely removed, this has meant a lot of work for Thunderbird to 
>>>> keep things going for add-ons. For the server side it has also 
>>>> meant a lot of extra work (addons.thunderbird.net is a fork of 
>>>> addons.mozilla.org). Add-on developers haven't had an easy ride 
>>>> either: The number of changes to make an add-on compatible has been 
>>>> significant.
>>>>
>>>> Going forwards we want to change this. Support for traditional 
>>>> add-ons is going to be dropped as soon as we're ready to do so 
>>>> internally. There are a few pieces of code that we need to convert 
>>>> over internally:
>>>>
>>>>   * Lightning: to be integrated into the code base
>>>>   * Mozmill (used in our test infra): we're converting over to using
>>>>     mochitests instead
>>>>
>>>> It's not yet clear exactly when we're ready to rip out the support 
>>>> for traditional add-ons from the code base, but it should be whitin 
>>>> the Thunderbird 72 time frame - so by end of 2019. The next major 
>>>> version of Thunderbird, version 78, will be out around June 2020. 
>>>> Up until then, code wise many things are going to change. For 
>>>> instance, what is left of XUL will be gradually going away, and 
>>>> documents will shifted to being XHTML with a less and less XUL flavor.
>>>>
>>>> Dropping support for non-MailExtension add-ons is also needed for 
>>>> addons.thunderbird.net. Supporting old-style add-ons would require 
>>>> a significant investment in the back-end there, since the Django 
>>>> version of the back-end would reach EOL and have to go through a 
>>>> painful and expensive upgrade.
>>>>
>>>> As an author of a traditional add-on, what should you do? There are 
>>>> two routes: A) convert your add-on to a MailExtension. If the API 
>>>> you need doesn't exist yet, tell us about it 
>>>> <https://bugzilla.mozilla.org/enter_bug.cgi?product=Thunderbird&component=General>. 
>>>> B) convert your add-on to a Web Extension Experiment 
>>>> <https://thunderbird-webextensions.readthedocs.io/en/68/how-to/experiments.html>. 
>>>> Most add-ons should be able to be converted to an experiment with a 
>>>> reasonable effort. The recommended path is forward is to convert it 
>>>> to an a MailExtension though. That will make sure the add-on works 
>>>> without significant changes over many years. If you go with option 
>>>> B, you'll have to maintain a lot of more code yourself and 
>>>> breakages can and will be bad unless you keep up really close. 
>>>> MailExtension Experiments should be seen as such, experiments with 
>>>> the goal of getting the API they need into Thunderbird core. Please 
>>>> work with us on getting the needed pieces in as a supported API. 
>>>> Initially we'll be allowing experiments to be exposed to the 
>>>> general public, but over time (years) Thunderbird will gravitate 
>>>> towards not having the experiments available to the general public, 
>>>> the same way it works for Firefox.
>>>>
>>>>  -Magnus
>>>>
>>>>
>>>> _______________________________________________
>>>> tb-planning mailing list
>>>> tb-planning at mozilla.org
>>>> https://mail.mozilla.org/listinfo/tb-planning
>>>
>>>
>>> _______________________________________________
>>> tb-planning mailing list
>>> tb-planning at mozilla.org
>>> https://mail.mozilla.org/listinfo/tb-planning
>>
>> _______________________________________________
>> tb-planning mailing list
>> tb-planning at mozilla.org
>> https://mail.mozilla.org/listinfo/tb-planning
>
>
> _______________________________________________
> tb-planning mailing list
> tb-planning at mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning


More information about the tb-planning mailing list