From mozmill to mochitests (Re: Intent to de-support: traditional add-ons)

ISHIKAWA,chiaki ishikawa at yk.rim.or.jp
Wed Oct 2 09:52:33 UTC 2019


I think somebody ought to raise a red flag here.

At last count, there were 32 test subdirectories and the total tests 
were somewhere in 1100-1200 range.
(A single file can contain multiple tests.)

I wonder who are working on the conversion aside from Geoff.

I didn't realize this 2019 timeline, BTW.
Schedule-wise, three months remain meaning more than 10 tests (in a few 
files) need to be converted daily.

I, for one, have looked at some tests before in order to understand why 
some tests fail due to my CPP file changes, but could not make head or 
tail of them at all, honestly speaking.
For example, I am trying to figure out why my local patches fail the 
following test, but
I could not fathom exactly what it is trying to do. Too sparse 
documentation of mozmill, I am afraid.

comm/mailnews/base/test/unit/test_quarantineFilterMove.js


I wonder if there are enough man-power for the conversion in time.
I am happy to be corrected if I am wrong.

If mochitests is easier to understand than mozmill, that would be super 
in the end for extending tests in general and adding new tests: there 
are many things that don't get checked properly  in current mozmill 
test. For example, I notice that some tests are not handling error 
dialog correctly at all. They either look at incorrect dialog and 
decided that the error is handled or never expected to encounter extra 
error dialogs at all.
In fact, there are cases where incorrect error dialog seems to be 
processed as the expected one  and the other
error dialog is simply ignored and timed out if I am not mistaken. You 
can look at the mozmill test screen and see such strange error dialog 
shown on the screen for timeout period and the test basically pauses 
until timeout kicks in and only then whlole test proceeds again.
The whole mozmill test takes less than two hours now on my Ryzen 1700 
CPU and so
you can simply run the test in the background and make the screen 
visible, and then do some other work on PC.
You will eventually notice that there are moments of frozen screen with 
error dialog being shown and nothing happens. It is hard to miss.

TIA

Chiaki


On 2019/10/02 18:06, Magnus Melin wrote:
>
> Please see bug 1571681 
> <https://bugzilla.mozilla.org/show_bug.cgi?id=1571681> tracking this 
> conversion.
>
> In bug 1585162 
> <https://bugzilla.mozilla.org/show_bug.cgi?id=1585162>there is already 
> a patch converting over all the calendar mozmill tests, probably 
> landing soon. You can see the approach from there.
>
> The timeline should within this 2019.
>
>  -Magnus
>
> On 02-10-2019 12:00, ISHIKAWA,chiaki wrote:
>> I have changed the subject line.
>>
>> I am focusing on a side topic.
>>
>> > Mozmill (used in our test infra): we're converting over to using 
>> mochitests instead
>>
>> So we are moving from mozmill to mochitests.
>>
>> How is it planned?
>>
>> I would like to see a smooth transition.
>>
>> For example,
>>
>> - rewriting test one by one from mozmill to mochitests and
>>   create a duplicate set of existing tests (functional duplicates) so 
>> that
>>   equivalent of all the tests are available and have been thoroughly 
>> tested
>>   by the time we switch from mozmill to mochitests completely.
>>
>> - Is there a document to describe how to rewrite mozmill tests to 
>> mochitests to guide such conversion?
>>
>> (- One thing I noticed is that there *ARE* xpcshell tests for TB, but 
>> xpcshell test framework doesn't create
>>    visible windows during tests and that cause some local hacks to 
>> fail since
>>    I could not produce visible error message to the user due to lack 
>> of visible screen
>>    when an serious I/O error occurs. I hope mochitests won't have 
>> such strange restriction.
>>    Oh well, since it checks the GUI operation as well, there WILL be 
>> screens, I suppose.)
>>
>> What is the planned timescale of the transition?
>> End of 2019 is unattainable.
>> Maybe the end of 2020?
>>
>> TIA
>>
>> Chiaki
>>
>>
>>
>> On 2019/10/01 22:02, Magnus Melin wrote:
>>>
>>> Since version 57, Firefox only supports add-ons through the 
>>> WebExtensions APIs. At the time, Thunderbird decided to continue 
>>> supporting traditional add-ons, since we hadn't yet been able to 
>>> develop replacing APIs for add-on developers to use.
>>>
>>> Since then, Thunderbird has been developing WebExtensions APIs (aka 
>>> MailExtensions), and the number of APIs available is continuously 
>>> growing: https://thunderbird-webextensions.readthedocs.io/
>>>
>>> Because the toolkit support for traditional add-ons has been largely 
>>> removed, this has meant a lot of work for Thunderbird to keep things 
>>> going for add-ons. For the server side it has also meant a lot of 
>>> extra work (addons.thunderbird.net is a fork of addons.mozilla.org). 
>>> Add-on developers haven't had an easy ride either: The number of 
>>> changes to make an add-on compatible has been significant.
>>>
>>> Going forwards we want to change this. Support for traditional 
>>> add-ons is going to be dropped as soon as we're ready to do so 
>>> internally. There are a few pieces of code that we need to convert 
>>> over internally:
>>>
>>>   * Lightning: to be integrated into the code base
>>>   * Mozmill (used in our test infra): we're converting over to using
>>>     mochitests instead
>>>
>>> It's not yet clear exactly when we're ready to rip out the support 
>>> for traditional add-ons from the code base, but it should be whitin 
>>> the Thunderbird 72 time frame - so by end of 2019. The next major 
>>> version of Thunderbird, version 78, will be out around June 2020. Up 
>>> until then, code wise many things are going to change. For instance, 
>>> what is left of XUL will be gradually going away, and documents will 
>>> shifted to being XHTML with a less and less XUL flavor.
>>>
>>> Dropping support for non-MailExtension add-ons is also needed for 
>>> addons.thunderbird.net. Supporting old-style add-ons would require a 
>>> significant investment in the back-end there, since the Django 
>>> version of the back-end would reach EOL and have to go through a 
>>> painful and expensive upgrade.
>>>
>>> As an author of a traditional add-on, what should you do? There are 
>>> two routes: A) convert your add-on to a MailExtension. If the API 
>>> you need doesn't exist yet, tell us about it 
>>> <https://bugzilla.mozilla.org/enter_bug.cgi?product=Thunderbird&component=General>. 
>>> B) convert your add-on to a Web Extension Experiment 
>>> <https://thunderbird-webextensions.readthedocs.io/en/68/how-to/experiments.html>. 
>>> Most add-ons should be able to be converted to an experiment with a 
>>> reasonable effort. The recommended path is forward is to convert it 
>>> to an a MailExtension though. That will make sure the add-on works 
>>> without significant changes over many years. If you go with option 
>>> B, you'll have to maintain a lot of more code yourself and breakages 
>>> can and will be bad unless you keep up really close. MailExtension 
>>> Experiments should be seen as such, experiments with the goal of 
>>> getting the API they need into Thunderbird core. Please work with us 
>>> on getting the needed pieces in as a supported API. Initially we'll 
>>> be allowing experiments to be exposed to the general public, but 
>>> over time (years) Thunderbird will gravitate towards not having the 
>>> experiments available to the general public, the same way it works 
>>> for Firefox.
>>>
>>>  -Magnus
>>>
>>>
>>> _______________________________________________
>>> tb-planning mailing list
>>> tb-planning at mozilla.org
>>> https://mail.mozilla.org/listinfo/tb-planning
>>
>>
>> _______________________________________________
>> tb-planning mailing list
>> tb-planning at mozilla.org
>> https://mail.mozilla.org/listinfo/tb-planning
>
> _______________________________________________
> tb-planning mailing list
> tb-planning at mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning




More information about the tb-planning mailing list