Upcoming changes to Thunderbird's tests

ISHIKAWA,chiaki ishikawa at yk.rim.or.jp
Thu Dec 5 19:36:03 UTC 2019

Dear Geoff,

Thank you for clarification.

So no change is necessary for trychooser option.

I will tinker with the valgrind test even then.


On 2019/12/05 11:29, Geoff Lankow wrote:
> On 05/12/2019 14:48, ISHIKAWA,chiaki wrote:
>> Dear Geoff,
>> I have been running mozmill and xpcshell tests on tryserver for quite 
>> sometime (and actually locally much more often, too).
>> >  * run both sets of tests in your Try runs
>> What would be the correct trychooser option to do this?
>> My current trychooser string is as follows. I test debug and optimize 
>> versions.
>> try: -b do -p linux64,macosx64,win64,win32 -u all -t none
> This will run the mochitests.
>> Also, can you enlighten me how to run the local mochtest of TB under 
>> linux on my PC?
>> (Is there a document regarding running TB test in mochitest framework?
>> I think there is a document for mochitest for FF. But for TB?
> This is the same as it is for Firefox: mach mochitest path/to/the/test
>> I would love to learn how to invoke TB binary under valgrind during 
>> local mochtest execution.
> The MDN docs 
> <https://developer.mozilla.org/en-US/docs/Mozilla/Testing/Valgrind> 
> suggest you can do exactly this. I know almost nothing about Valgrind 
> so I really can't comment.
> I do know that Firefox's build infrastructure sometimes creates a 
> Valgrind job so it's probably possible to do the same in Thunderbird, 
> but again, I really don't know.
>> Many years ago, when I began hacking on TB code after getting bitten 
>> by its serious bugs such as
>> not checking file I/O error code while compacting a folder [I lost 
>> half my folder due to file system overflow on a note PC with only 
>> 20GB of disk back then.],
>> I could not fathom how to run TB under valgrind during |make 
>> mozmill|, and so
>> I simply decided to substitute |thunderbird| executable under linux 
>> with a wrapper program to
>> run |thunderbird-bin| executable under valgrind with suitable 
>> options. Works like a charm. I have found maybe a dozen or so memory 
>> access issues thanks to this.
>> Chiaki
>> PS: I have found a few seriously looking uninitialized memory 
>> accesses after I invoked xpcshell tests under valgrind locally under 
>> linux lately.
>> (Takes 20+ hours to run ...)
>> Initially, I could not believe such serious issues and thought they 
>> were valgrind's false positives.
>> But I have confirmed they are for real to my amazement.
>> (I have already reported the obvious bugs to bugzilla. However, some 
>> issues were hard to believe initially and I have not reported them in 
>> detail yet.)
>> They are xpcshell tests, and so luckily I should not need to change 
>> the test  .js files  under mozmil.: I am inserting a few dump 
>> statements so that if the bug is triggered on tryserver [again after 
>> proper C++ code fix], it will be noticeable by error messages.
>> Of course, I certainly need to change C++ code and am investigating 
>> what I should do.  Once I learn more details I will post the issue 
>> and my observation to bugzilla to learn what will be the correct fix...
>> Being able to run TB binary under valgrind during mochitest is an 
>> essential feature for long term maintenance of TB code IMHO.
>> Of course, somebody HAS TO LOOK AT the test result periodically.
>> On 2019/12/03 11:08, Geoff Lankow wrote:
>>> Hello everybody
>>> Because of the changing add-on environment, we plan to switch off 
>>> Mozmill testing by the end of the year. Mozmill ran using a 
>>> bootstrapped extension (already hacked from a non-bootstrapped 
>>> extension) which no longer be feasible when we remove support for 
>>> bootstrapped extensions.
>>> Shortly I'll be landing a patch which /copies/ all Mozmill tests to 
>>> Mochitest (bug 1599991 
>>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1599991>). Some of 
>>> them are switched off because they fail reliably.
>>> For a time both sets of tests will be running simultaneously, so you 
>>> will need to:
>>>  * run both sets of tests in your Try runs
>>>  * if you break something, check it is fixed in both tests
>>>  * modify both copies of a test if you make a modification
>>>  * there may be some lingering orange on TreeHerder for a while – if
>>>    your Try run is orange and you don't know if you caused it, ask
>>> The new tests will live under mail/test/browser with similar names 
>>> to the old tests in mail/test/mozmill. The modules in 
>>> mail/test/mozmill/shared-modules are used by both sets of tests (ie. 
>>> there's only one copy of these files).
>>> Once we're satisfied the new tests are working okay, Mozmill will be 
>>> switched off forever and you can unlearn make -C path/to/objdir 
>>> mozmill-one SOLO_TEST=folder/test-name.js :-)
>>> GL
>>> P.S. Mochitest debug runs come with leak checking turned on by 
>>> default. Some of the test runs leak over 100MB. This is not good! 
>>> Please help fix it if you have the know-how.
>>> _______________________________________________
>>> tb-planning mailing list
>>> tb-planning at mozilla.org
>>> https://mail.mozilla.org/listinfo/tb-planning
>> _______________________________________________
>> tb-planning mailing list
>> tb-planning at mozilla.org
>> https://mail.mozilla.org/listinfo/tb-planning
> _______________________________________________
> tb-planning mailing list
> tb-planning at mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning

More information about the tb-planning mailing list