quasi-survey on writing mozmill tests for thunderbird
ben.bucksch at beonex.com
Thu Mar 17 19:45:54 UTC 2011
On 17.03.2011 20:28, Andrew Sutherland wrote:
> On 03/17/2011 11:56 AM, Ben Bucksch wrote:
>> And, FWIW, [nested event loop of fakeserver, instead of threading]
>> was the other reason why the test framework was unusable for me, it
>> took me *days* (where I lost lots of hair) to get my test half-way
>> working because of this, and the tests I wrote are still failing
>> randomly until this day.
> Can you elaborate on this? Was the problem the nested event loop spinning
Yes... This fact produced non-deterministic results. Things worked most
of the time for me, but apparently not for others, or not always. And
when I changed something entirely unrelated, things related to the
fakeserver broke again. A total nightmare.
FWIW, it were the tests for
<https://bugzilla.mozilla.org/show_bug.cgi?id=525238>. I spend 1.5 weeks
just for the tests.
It also makes for a totally illogical API (when the server actually
starts doing work), and lots of hacks (strange commands to spin the
loop, waits etc.) in many existing tests to make it working.
After a long time of debugging, I realized that it's due to the event
loop. I had assumed that the fakeserver runs in a thread (or process),
because that's for me the only way to do it.
I don't really understand why it's such a problem either. You don't need
to share data between client and server (a real client doesn't share
memory with the server either!), you just pass the state and behaviour
in before you start the server (thread), and then optionally at the end
(when the server quits) pass some data (e.g. which commands were
received) back to the test. For consecutive test runs against different
server variations (I have that in my tests, too), we'd just shut down
and start a new server.
More information about the tb-planning