Re: Measuring Australis…

Ehsan Akhgari ehsan at mozilla.com
Thu Oct 17 03:30:14 UTC 2013


One question that I would like to see on this list is whether people's
behavior changes after switching to the new theme, i.e., do they manage to
adapt their known workflows and mussel memory through the transition after
learning about the new ways of doing things. That requires being able to
correlate between different telemetry submissions of course which may not
be easy.

Cheers,
Ehsan
On 2013-10-16 10:13 PM, "Blake Winton" <bwinton at mozilla.com> wrote:

> Hello firefox-dev-ians,
>
> I've been chatting with some people on the Firefox UX team, and we have
> some questions about the impact the new Australis re-design will have on
> our users.  I mean, we've put a lot of work into the design of Australis,
> and we have some goals for how we want it to change user behaviour, but we
> don't *know* whether people will react to it in the way we expect.  I
> also believe that the engineers who have been (and are still) putting a ton
> of work into landing the new design in a performant way are interested in
> how their hard work is impacting the people who use Firefox.  Finally, I
> believe that with a set of Telemetry probes, we can get the answers to most
> of those questions.
>
> The list of questions<https://firefox-ux.etherpad.mozilla.org/measuringaustralis> we
> have come up with so far is:
>
>
>    - Do people use the Menu Panel?
>
>
>    - vs. how many people used the orange button?
>
>
>    - What do people use in the Menu Panel?
>
>
>    - Do they use new tools that they didn't use before
>
>
>    - do rates of usage increase/decrease vs. pre-australis
>
>
>    - Do people enter Customize Mode?
>
>
>    - Do people actually customize things?
>
>
>    - What do they add/remove?
>
>
>    - How many people click "Restore Defaults"?
>
>
>    - How do people exit Customize Mode?
>
>
>    - Do people get stuck in customize mode?
>
>
>    - Do people use the Bookmark Star?
>
>
>    - Are they bookmarking more or less?
>
>
>    - Do they use the Bookmark dropdown?
>
>
>    - any change in usage across the different ways of getting at
>    bookmarks?
>
>
>    - What do users do upon seeing the First Run page?
>
>
>    - Do users use what we're showing them on the First Run page?
>
>
>    - Do people have more tabs?
>
>
>    - Do people switch between tabs more?
>
>
>    - Can people find the right background tab now that they're
>    deemphasized?
>
>
>    - How do the answers to all of the above questions change after a
>    week?  A month?
>
>
>    - Are people using find-in-page more or less since Australis?
>
>
>    - Ditto for everything else in the panel.
>
>
>    - Are the tab areas discriminable?
>
>
>    - How often do people click on a tab, and then click on a different
>    one?
>
>
>    - Usage increase or decrease of the Menu (OS X and Linux! ;)
>
>
>    - Are people pleased with the new look and feel?
>
>
>    - Does this have any effect on add-ons?
>
>
>    - Will it break people?
>
>
>    - Where the [redacted] is the add-on bar?
>
>
>    - Are people dragging windows more or less?  (We're reducing titlebar
>    space…)
>
>
>    - Does the rate of people detaching tabs change?
>
>
>    - Are people more angry with us/Firefox?
>
>
>    - We can use Input/Feedback for this.
>
>
>    - Are people using more lightweight themes?
>
>
>    - How do people like the new style?
>
>
>    - This probably needs a survey, which isn't really Telemetry…
>
>
> I suspect there are more things we are interested in, and that some of
> these things aren't able to be measured through Telemetry, so any comments
> or additions are more than welcome!
>
> So, my requests to you are as follows:
>
>    1. What am I missing in that list that we want to know or are
>    interested in?
>    2. Which items can't we get data for with Telemetry?  (Bonus points if
>    you can suggest an alternate way of answering the question!)
>    3. For the items we can measure with Telemetry, which probes do we
>    need to add?
>    4. Am I totally off-base here?  Does this sound reasonable to
>    everyone?  Does it make sense at all?  (Maybe this should be #1… ;)
>
>
> Assuming this is something that we all feel is valuable, then once we have
> a reasonably final list, I'll be happy to file bugs and meta-bugs, and
> (once we have some data) talk to Metrics to figure out what the data is
> showing us.  Heck, if it's not too hard to figure out, I'ld even be up for writing
> some of the probes<https://developer.mozilla.org/en-US/docs/Performance/Adding_a_new_Telemetry_probe>,
> but I suspect that there's more to do than I can do by myself, and so any
> assistance would be welcome!
>
> Thank you all for your input,
> Blake.
> --
> Blake Winton   UX Engineer
> bwinton at mozilla.com
>
>
> _______________________________________________
> firefox-dev mailing list
> firefox-dev at mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20131016/3a8eab2f/attachment.html>


More information about the firefox-dev mailing list