Autophone Update - 2014-02-05

Bob Clary bob at bclary.com
Thu Feb 6 10:46:50 PST 2014


In bug 852264 [1] I landed a change in the profile location from the 
sdcard to /data/local/ which will hopefully improve the reliability of 
the throbber measurements. This change affects test runs for builds 
beginning with  2014-02-05.

You will notice an apparent improvement for some of the 
devices/measurements. I believe this is mainly due to reduced standard 
deviation in the measurements and is not necessarily due to a real 
performance difference. If you are interested in the change in behavior 
I did a comparison [2] of before and after behavior for two devices.

I am planning on another change in bug 968920 [3] to use the standard 
error percentage as a discriminator in determining the reliability of a 
test run. If you have any opinions about this proposed change, please 
comment in the bug.

Another possible change I am considering is moving the local test files 
for devices with internal sdcards to an external sdcard. Once we have 
some experience with the new behavior I will evaluate whether this would 
improve reliability.

With the number of devices and repositories being tested, the phonedash 
[4] ui is becoming more difficult to use. If you are interested, bug 
967052 [5] is tracking the work to make the ui more user friendly.

/bc

[1] <https://bugzilla.mozilla.org/show_bug.cgi?id=852264>
[2] <http://people.mozilla.org/~bclary/log/2014/02/02/autophone/index.html>
[3] <https://bugzilla.mozilla.org/show_bug.cgi?id=968920>
[4] <http://phonedash.mozilla.org/>
[5] <https://bugzilla.mozilla.org/show_bug.cgi?id=967052>


More information about the mobile-firefox-dev mailing list