quasi-survey on writing mozmill tests for thunderbird

Ben Bucksch ben.bucksch at beonex.com
Thu Mar 17 19:45:54 UTC 2011

On 17.03.2011 20:28, Andrew Sutherland wrote:
> On 03/17/2011 11:56 AM, Ben Bucksch wrote:
>> And, FWIW, [nested event loop of fakeserver, instead of threading] 
>> was the other reason why the test framework was unusable for me, it 
>> took me *days* (where I lost lots of hair) to get my test half-way 
>> working because of this, and the tests I wrote are still failing 
>> randomly until this day.
> Can you elaborate on this?  Was the problem the nested event loop spinning

Yes... This fact produced non-deterministic results. Things worked most 
of the time for me, but apparently not for others, or not always. And 
when I changed something entirely unrelated, things related to the 
fakeserver broke again. A total nightmare.

FWIW, it were the tests for 
<https://bugzilla.mozilla.org/show_bug.cgi?id=525238>. I spend 1.5 weeks 
just for the tests.

It also makes for a totally illogical API (when the server actually 
starts doing work), and lots of hacks (strange commands to spin the 
loop, waits etc.) in many existing tests to make it working.

After a long time of debugging, I realized that it's due to the event 
loop. I had assumed that the fakeserver runs in a thread (or process), 
because that's for me the only way to do it.

I don't really understand why it's such a problem either. You don't need 
to share data between client and server (a real client doesn't share 
memory with the server either!), you just pass the state and behaviour 
in before you start the server (thread), and then optionally at the end 
(when the server quits) pass some data (e.g. which commands were 
received) back to the test. For consecutive test runs against different 
server variations (I have that in my tests, too), we'd just shut down 
and start a new server.

More information about the tb-planning mailing list