Autophone Update 2014-05-03

Bob Clary bob at bclary.com
Sat May 3 04:31:15 PDT 2014


As part of Bug 990601 - Autophone - use adb instead of tcp/ip to control 
devices <https://bugzilla.mozilla.org/show_bug.cgi?id=990601>, I've 
updated the production Autophone Mac Mini server with the second wip 
patch, have updated the device names to be more descriptive and have 
started running tests for 2014-05-03 and later.

I've added back some of the non-nexus devices for side by side 
comparison of their relative reproducibility. I am running builds for 
all three repositories for each device and am not running the 
Mochitest-Canvas tests. This will give a better 
cross-device/cross-repository comparison of the results and will help us 
gauge the reliability of the measurements.

You will note a 'discontinuous' change from 2014-05-02 to 2014-05-03. 
This is due to the change in how the fennec start time is detected now 
that the SUTAgent is no longer being used and should not be considered a 
real performance change.

You can follow along at 
<http://phonedash.mozilla.org/#/org.mozilla.fennec/totalthrobber/local-twitter/norejected/2014-05-01/2014-05-09/notcached/errorbars/standarderror>.

I will be taking Autophone offline sometime next week and will be 
shipping the server and phones to Mountain View to be installed in the 
QA lab in the new office. It will be offline until sometime in the 
middle of the week of May 12.

/bc


More information about the mobile-firefox-dev mailing list