Measuring Australis…

Blake Winton bwinton at mozilla.com
Thu Oct 17 02:13:10 UTC 2013


Hello firefox-dev-ians,

I've been chatting with some people on the Firefox UX team, and we have some questions about the impact the new Australis re-design will have on our users.  I mean, we've put a lot of work into the design of Australis, and we have some goals for how we want it to change user behaviour, but we don't know whether people will react to it in the way we expect.  I also believe that the engineers who have been (and are still) putting a ton of work into landing the new design in a performant way are interested in how their hard work is impacting the people who use Firefox.  Finally, I believe that with a set of Telemetry probes, we can get the answers to most of those questions.

The list of questions we have come up with so far is:

Do people use the Menu Panel?
vs. how many people used the orange button?
What do people use in the Menu Panel?
Do they use new tools that they didn't use before
do rates of usage increase/decrease vs. pre-australis
Do people enter Customize Mode?
Do people actually customize things?
What do they add/remove?
How many people click "Restore Defaults"?
How do people exit Customize Mode?
Do people get stuck in customize mode?
Do people use the Bookmark Star?
Are they bookmarking more or less?
Do they use the Bookmark dropdown?
any change in usage across the different ways of getting at bookmarks?
What do users do upon seeing the First Run page?
Do users use what we're showing them on the First Run page?
Do people have more tabs?
Do people switch between tabs more?
Can people find the right background tab now that they're deemphasized?
How do the answers to all of the above questions change after a week?  A month?
Are people using find-in-page more or less since Australis?
Ditto for everything else in the panel.
Are the tab areas discriminable?
How often do people click on a tab, and then click on a different one?
Usage increase or decrease of the Menu (OS X and Linux! ;)
Are people pleased with the new look and feel?
Does this have any effect on add-ons?
Will it break people?
Where the [redacted] is the add-on bar?
Are people dragging windows more or less?  (We're reducing titlebar space…)
Does the rate of people detaching tabs change?
Are people more angry with us/Firefox?
We can use Input/Feedback for this.
Are people using more lightweight themes?
How do people like the new style?
This probably needs a survey, which isn't really Telemetry…

I suspect there are more things we are interested in, and that some of these things aren't able to be measured through Telemetry, so any comments or additions are more than welcome!

So, my requests to you are as follows:
What am I missing in that list that we want to know or are interested in?
Which items can't we get data for with Telemetry?  (Bonus points if you can suggest an alternate way of answering the question!)
For the items we can measure with Telemetry, which probes do we need to add?
Am I totally off-base here?  Does this sound reasonable to everyone?  Does it make sense at all?  (Maybe this should be #1… ;)

Assuming this is something that we all feel is valuable, then once we have a reasonably final list, I'll be happy to file bugs and meta-bugs, and (once we have some data) talk to Metrics to figure out what the data is showing us.  Heck, if it's not too hard to figure out, I'ld even be up for writing some of the probes, but I suspect that there's more to do than I can do by myself, and so any assistance would be welcome!

Thank you all for your input,
Blake.
-- 
Blake Winton   UX Engineer
bwinton at mozilla.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20131016/6b595db0/attachment.html>


More information about the firefox-dev mailing list