Upcoming changes to Thunderbird's tests

Geoff Lankow geoff at thunderbird.net
Thu Dec 5 02:29:02 UTC 2019

On 05/12/2019 14:48, ISHIKAWA,chiaki wrote:
> Dear Geoff,
> I have been running mozmill and xpcshell tests on tryserver for quite 
> sometime (and actually locally much more often, too).
> >  * run both sets of tests in your Try runs
> What would be the correct trychooser option to do this?
> My current trychooser string is as follows. I test debug and optimize 
> versions.
> try: -b do -p linux64,macosx64,win64,win32 -u all -t none

This will run the mochitests.

> Also, can you enlighten me how to run the local mochtest of TB under 
> linux on my PC?
> (Is there a document regarding running TB test in mochitest framework?
> I think there is a document for mochitest for FF. But for TB?
This is the same as it is for Firefox: mach mochitest path/to/the/test
> I would love to learn how to invoke TB binary under valgrind during 
> local mochtest execution.

The MDN docs 
suggest you can do exactly this. I know almost nothing about Valgrind so 
I really can't comment.

I do know that Firefox's build infrastructure sometimes creates a 
Valgrind job so it's probably possible to do the same in Thunderbird, 
but again, I really don't know.

> Many years ago, when I began hacking on TB code after getting bitten 
> by its serious bugs such as
> not checking file I/O error code while compacting a folder [I lost 
> half my folder due to file system overflow on a note PC with only 20GB 
> of disk back then.],
> I could not fathom how to run TB under valgrind during |make mozmill|, 
> and so
> I simply decided to substitute |thunderbird| executable under linux 
> with a wrapper program to
> run |thunderbird-bin| executable under valgrind with suitable options. 
> Works like a charm. I have found maybe a dozen or so memory access 
> issues thanks to this.
> Chiaki
> PS: I have found a few seriously looking uninitialized memory accesses 
> after I invoked xpcshell tests under valgrind locally under linux lately.
> (Takes 20+ hours to run ...)
> Initially, I could not believe such serious issues and thought they 
> were valgrind's false positives.
> But I have confirmed they are for real to my amazement.
> (I have already reported the obvious bugs to bugzilla. However, some 
> issues were hard to believe initially and I have not reported them in 
> detail yet.)
> They are xpcshell tests, and so luckily I should not need to change 
> the test  .js files  under mozmil.: I am inserting a few dump 
> statements so that if the bug is triggered on tryserver [again after 
> proper C++ code fix], it will be noticeable by error messages.
> Of course, I certainly need to change C++ code and am investigating 
> what I should do.  Once I learn more details I will post the issue and 
> my observation to bugzilla to learn what will be the correct fix...
> Being able to run TB binary under valgrind during mochitest is an 
> essential feature for long term maintenance of TB code IMHO.
> Of course, somebody HAS TO LOOK AT the test result periodically.
> On 2019/12/03 11:08, Geoff Lankow wrote:
>> Hello everybody
>> Because of the changing add-on environment, we plan to switch off 
>> Mozmill testing by the end of the year. Mozmill ran using a 
>> bootstrapped extension (already hacked from a non-bootstrapped 
>> extension) which no longer be feasible when we remove support for 
>> bootstrapped extensions.
>> Shortly I'll be landing a patch which /copies/ all Mozmill tests to 
>> Mochitest (bug 1599991 
>> <https://bugzilla.mozilla.org/show_bug.cgi?id=1599991>). Some of them 
>> are switched off because they fail reliably.
>> For a time both sets of tests will be running simultaneously, so you 
>> will need to:
>>  * run both sets of tests in your Try runs
>>  * if you break something, check it is fixed in both tests
>>  * modify both copies of a test if you make a modification
>>  * there may be some lingering orange on TreeHerder for a while – if
>>    your Try run is orange and you don't know if you caused it, ask
>> The new tests will live under mail/test/browser with similar names to 
>> the old tests in mail/test/mozmill. The modules in 
>> mail/test/mozmill/shared-modules are used by both sets of tests (ie. 
>> there's only one copy of these files).
>> Once we're satisfied the new tests are working okay, Mozmill will be 
>> switched off forever and you can unlearn make -C path/to/objdir 
>> mozmill-one SOLO_TEST=folder/test-name.js :-)
>> GL
>> P.S. Mochitest debug runs come with leak checking turned on by 
>> default. Some of the test runs leak over 100MB. This is not good! 
>> Please help fix it if you have the know-how.
>> _______________________________________________
>> tb-planning mailing list
>> tb-planning at mozilla.org
>> https://mail.mozilla.org/listinfo/tb-planning
> _______________________________________________
> tb-planning mailing list
> tb-planning at mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/tb-planning/attachments/20191205/83d47efb/attachment.html>

More information about the tb-planning mailing list