Upcoming changes to Thunderbird's tests

ISHIKAWA,chiaki ishikawa at yk.rim.or.jp
Thu Dec 5 01:48:11 UTC 2019

Dear Geoff,

I have been running mozmill and xpcshell tests on tryserver for quite 
sometime (and actually locally much more often, too).

 >  * run both sets of tests in your Try runs

What would be the correct trychooser option to do this?
My current trychooser string is as follows. I test debug and optimize 

try: -b do -p linux64,macosx64,win64,win32 -u all -t none

Also, can you enlighten me how to run the local mochtest of TB under 
linux on my PC?
(Is there a document regarding running TB test in mochitest framework?
I think there is a document for mochitest for FF. But for TB?

I would love to learn how to invoke TB binary under valgrind during 
local mochtest execution.

Many years ago, when I began hacking on TB code after getting bitten by 
its serious bugs such as
not checking file I/O error code while compacting a folder [I lost half 
my folder due to file system overflow on a note PC with only 20GB of 
disk back then.],
I could not fathom how to run TB under valgrind during |make mozmill|, 
and so
I simply decided to substitute |thunderbird| executable under linux with 
a wrapper program to
run |thunderbird-bin| executable under valgrind with suitable options. 
Works like a charm. I have found maybe a dozen or so memory access 
issues thanks to this.


PS: I have found a few seriously looking uninitialized memory accesses 
after I invoked xpcshell tests under valgrind locally under linux lately.
(Takes 20+ hours to run ...)
Initially, I could not believe such serious issues and thought they were 
valgrind's false positives.
But I have confirmed they are for real to my amazement.
(I have already reported the obvious bugs to bugzilla. However, some 
issues were hard to believe initially and I have not reported them in 
detail yet.)
They are xpcshell tests, and so luckily I should not need to change the 
test  .js files  under mozmil.: I am inserting a few dump statements so 
that if the bug is triggered on tryserver [again after proper C++ code 
fix], it will be noticeable by error messages.
Of course, I certainly need to change C++ code and am investigating what 
I should do.  Once I learn more details I will post the issue and my 
observation to bugzilla to learn what will be the correct fix...

Being able to run TB binary under valgrind during mochitest is an 
essential feature for long term maintenance of TB code IMHO.
Of course, somebody HAS TO LOOK AT the test result periodically.

On 2019/12/03 11:08, Geoff Lankow wrote:
> Hello everybody
> Because of the changing add-on environment, we plan to switch off 
> Mozmill testing by the end of the year. Mozmill ran using a 
> bootstrapped extension (already hacked from a non-bootstrapped 
> extension) which no longer be feasible when we remove support for 
> bootstrapped extensions.
> Shortly I'll be landing a patch which /copies/ all Mozmill tests to 
> Mochitest (bug 1599991 
> <https://bugzilla.mozilla.org/show_bug.cgi?id=1599991>). Some of them 
> are switched off because they fail reliably.
> For a time both sets of tests will be running simultaneously, so you 
> will need to:
>  * run both sets of tests in your Try runs
>  * if you break something, check it is fixed in both tests
>  * modify both copies of a test if you make a modification
>  * there may be some lingering orange on TreeHerder for a while – if
>    your Try run is orange and you don't know if you caused it, ask
> The new tests will live under mail/test/browser with similar names to 
> the old tests in mail/test/mozmill. The modules in 
> mail/test/mozmill/shared-modules are used by both sets of tests (ie. 
> there's only one copy of these files).
> Once we're satisfied the new tests are working okay, Mozmill will be 
> switched off forever and you can unlearn make -C path/to/objdir 
> mozmill-one SOLO_TEST=folder/test-name.js :-)
> GL
> P.S. Mochitest debug runs come with leak checking turned on by 
> default. Some of the test runs leak over 100MB. This is not good! 
> Please help fix it if you have the know-how.
> _______________________________________________
> tb-planning mailing list
> tb-planning at mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning

More information about the tb-planning mailing list