<div dir="ltr"><div>+1 for shipping the same UI to all Android versions.<br><br></div>From a UX point of view I can see many benefits to giving all our users the same UI/UX. Especially since L has support down to 2.3 (?) now.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 3, 2015 at 11:25 AM, <span dir="ltr"><<a href="mailto:mobile-firefox-dev-request@mozilla.org" target="_blank">mobile-firefox-dev-request@mozilla.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send mobile-firefox-dev mailing list submissions to<br>
<a href="mailto:mobile-firefox-dev@mozilla.org">mobile-firefox-dev@mozilla.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://mail.mozilla.org/listinfo/mobile-firefox-dev" rel="noreferrer" target="_blank">https://mail.mozilla.org/listinfo/mobile-firefox-dev</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:mobile-firefox-dev-request@mozilla.org">mobile-firefox-dev-request@mozilla.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:mobile-firefox-dev-owner@mozilla.org">mobile-firefox-dev-owner@mozilla.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of mobile-firefox-dev digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: Shipping one UI for all Android versions (Nicholas Alexander)<br>
2. A/B Testing in Firefox for Android (Mark Finkle)<br>
3. Re: A/B Testing in Firefox for Android (Martin Thomson)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Thu, 3 Sep 2015 10:46:02 -0700<br>
From: Nicholas Alexander <<a href="mailto:nalexander@mozilla.com">nalexander@mozilla.com</a>><br>
To: Sebastian Kaspari <<a href="mailto:s.kaspari@gmail.com">s.kaspari@gmail.com</a>><br>
Cc: mobile-firefox-dev <<a href="mailto:mobile-firefox-dev@mozilla.org">mobile-firefox-dev@mozilla.org</a>><br>
Subject: Re: Shipping one UI for all Android versions<br>
Message-ID:<br>
<<a href="mailto:CAMnWBR0k5OyFJ0DVUNzprx1NR7BN93T2u80aHPVr5wFd2TNVDg@mail.gmail.com">CAMnWBR0k5OyFJ0DVUNzprx1NR7BN93T2u80aHPVr5wFd2TNVDg@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi Sebastian,<br>
<br>
On Thu, Sep 3, 2015 at 2:11 AM, Sebastian Kaspari <<a href="mailto:s.kaspari@gmail.com">s.kaspari@gmail.com</a>><br>
wrote:<br>
<br>
> Hey everyone,<br>
><br>
> I remember when I joined Mozilla Michael asked me what's different working<br>
> here compared to working at other Android "companies". Back then my answer<br>
> was: The build system! Because this was so obviously different and maybe a<br>
> bit painful at the beginning. Since the build system gets better almost<br>
> every day (Thank you nalexander!) my answer today would be: The big effort<br>
> we are making to ship multiple variations of our UI to the different<br>
> Android versions.<br>
><br>
<br>
I don't have direct expertise in maintaining these UI variations, but I<br>
would like to pile on that this type of unification is a Good Thing, and<br>
I'm happy Sebastian has raised it.<br>
<br>
<br>
> Two examples of what I mean:<br>
><br>
> 1) The overflow menu. In Firefox: On most modern devices it's in the top<br>
> right corner. On Gingerbread it's a simple, old Android menu triggered by<br>
> pressing a hardware menu button. Then there are some devices with Android<br>
> 4+ and a menu button. These devices get a menu that shows up from the<br>
> bottom of the screen but has more features like the overflow menu. And I<br>
> wouldn't be surprised if there are even more variations. :) Touching this<br>
> code is not always easy. Writing UI tests can be frustrating because you<br>
> have to handle all these variations (even though we do a good job hiding<br>
> this behind components).<br>
><br>
> 2) Themes: We have a default theme that derives from Android's default<br>
> theme (Back then this could be anything, including funky variations shipped<br>
> by the device manufacturers), a v11+ theme that inherits from the Holo<br>
> theme, a v14+ style that overrides some styles and a v21+ theme that<br>
> inherits from the Material theme. In addition to that we define styles in<br>
> default, v11, v13, v16 and v19. Some styles are inherited from previous<br>
> versions and some are overridden. I recently tried to change the v21 theme<br>
> and I couldn't follow all style paths because of the complexity.<br>
><br>
> On the one hand, I'm impressed that we have been able to maintain this and<br>
> are shipping a Look & Feel that matches the one of the particular platform<br>
> very closely, but on the other hand, I don't know of any other popular app<br>
> that does that. Instead they ship the same (modern, now: Material) UI to<br>
> all devices with the help of the support library and appcompat library (And<br>
> nowadays the Android design support library too).<br>
><br>
> Some examples:<br>
> * The Toolbar API allows us to have the same ActionBar behavior<br>
> independently of the Android version we are running on (vs. maintaining<br>
> different menus)<br>
> * The appcompat library comes with a theme that resembles the Material<br>
> theme on all Android versions (as good as it can). This allows us to use<br>
> and adapt just one theme.<br>
> * We are maintaining some custom components that we could replace with<br>
> feature richer and more tested support components, e.g our<br>
> TabMenuStripLayout<br>
><br>
> But of course there are some downsides:<br>
> * We already include the support library and the appcompat library (at<br>
> least for builds with MOZ_NATIVE_DEVICES set) but not the design library<br>
> (bug 1189306). We don't need it yet but it will grow the APK size. Even<br>
> though proguard might be able to reduce this a bit.<br>
><br>
<br>
The AAR is 200k, of which almost all of that is classes.jar. It's worth<br>
investigating the impact of Proguard further, but using this design library<br>
sounds like a great idea. The more custom code we replace with upstream<br>
code, the easier it is for new engineers to come to the code base.<br>
<br>
<br>
> * Shipping the same thing to all devices with the help of support<br>
> components removes the advantage of possible APK size reductions by<br>
> splitting APKs by version.<br>
><br>
<br>
In some ways, yes. In others, no. We do need to be aggressive about<br>
guarding features across APK splits.<br>
<br>
<br>
> Okay, this mail got longer than I thought. I wanted to write it for some<br>
> time to hear your opinions but now that I saw a new bug about inheriting<br>
> styles from AppCompat (bug 1201206) I wanted to get it out. :)<br>
><br>
<br>
Thanks for this email. I hope that mcomella, mhaigh, and others with more<br>
informed opinions will weigh in here to set public direction.<br>
<br>
Nick<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://mail.mozilla.org/pipermail/mobile-firefox-dev/attachments/20150903/29bdcc33/attachment-0001.html" rel="noreferrer" target="_blank">http://mail.mozilla.org/pipermail/mobile-firefox-dev/attachments/20150903/29bdcc33/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Thu, 3 Sep 2015 14:16:42 -0400<br>
From: Mark Finkle <<a href="mailto:mfinkle@mozilla.com">mfinkle@mozilla.com</a>><br>
To: "<a href="mailto:mobile-firefox-dev@mozilla.org">mobile-firefox-dev@mozilla.org</a>" <<a href="mailto:mobile-firefox-dev@mozilla.org">mobile-firefox-dev@mozilla.org</a>>,<br>
"group, mozilla.dev.planning" <<a href="mailto:dev-planning@lists.mozilla.org">dev-planning@lists.mozilla.org</a>><br>
Subject: A/B Testing in Firefox for Android<br>
Message-ID:<br>
<<a href="mailto:CACfDpeYXmAcpyReYEdqUAJifNsotUDxaguMHpaZZY16fnG3v%2Bg@mail.gmail.com">CACfDpeYXmAcpyReYEdqUAJifNsotUDxaguMHpaZZY16fnG3v+g@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
We have decided to start running A/B Testing [1] in Firefox for Android.<br>
These experiments are intended to optimize specific outcomes, as well as,<br>
inform our long-term design decisions. We want to create the best Firefox<br>
experience we can, and these experiments will help.<br>
<br>
The system will also allow us to throttle the release of features, called<br>
staged rollout, so we can monitor new features in a controlled manner<br>
across a large userbase and a fragmented device ecosystem. If we need to<br>
rollback a feature for some reason, we'd have the ability to do that.<br>
<br>
Technical details:<br>
* Switchboard is used to control experiment segmenting and staged rollout.<br>
* Telemetry is used to collect metrics about an experiment.<br>
* FHR is used to track active experiments so we can correlate to<br>
application usage.<br>
<br>
== What is Switchboard? ==<br>
<br>
Switchboard is an open source SDK for doing A/B testing and staged rollouts<br>
[2][3]. It connects to a server component, which maintains a list of active<br>
experiments.<br>
<br>
The SDK does create a UUID, which is stored on the device. The UUID is sent<br>
to the server, which uses it to "bucket" the client, but the UUID is never<br>
stored on the server. In fact, the server does not store any data. The<br>
server we are using is being hosted by Mozilla.<br>
<br>
We decided to start using Switchboard because it's simple, open source,<br>
saves no data and can be hosted by Mozilla. Thanks to the KeepSafe folks<br>
for releasing Switchboard.<br>
<br>
== Planning Experiments ==<br>
<br>
The Mobile Product and UX teams are the primary drivers for creating<br>
experiments, but as is common on the Mobile team, ideas can come from<br>
anywhere. We have been working with the Mozilla Growth team, getting a<br>
better understanding of how to design the experiments and analyze the<br>
metrics. UX researchers will also have input into the experiments.<br>
<br>
Once Product and UX complete the experiment design, Development would land<br>
code in Firefox to implement the desired variations of the experiment.<br>
Development would also land code in the Switchboard server to control the<br>
configuration of the experiment: Is it active? How are the variations<br>
distributed across the user population?<br>
<br>
Since we use Telemetry to collect metrics on the experiments, the Beta<br>
channel is likely our best time period to run experiments. Telemetry is on<br>
by default on Nightly, Aurora and Beta; and Beta is the largest userbase of<br>
those three channels.<br>
<br>
Once we decide which variation of the experiment is the "winner", we'll<br>
change the Switchboard server configuration for the experiment so that 100%<br>
of the userbase will flow through the winning variation.<br>
<br>
Yes, a small percentage of the Release channel has Telemetry enabled, but<br>
it might be too small to be useful for experimentation. Time will tell.<br>
<br>
Note: Switchboard itself will be enabled on all channels. It collects no<br>
data and gives us a "code-free" way of staging rollouts. It much less risky<br>
and time consuming than uplifting patches that need to land on branches at<br>
specific times.<br>
<br>
== What's Happening Now? ==<br>
<br>
Switchboard has already landed in Nightly [4] and is currently behind a<br>
Nightly build flag. Once we feel comfortable, we'll let it ride the trains.<br>
Our first experiment will likely be testing a new onboarding experience [5].<br>
<br>
[1] <a href="https://en.wikipedia.org/wiki/A/B_testing" rel="noreferrer" target="_blank">https://en.wikipedia.org/wiki/A/B_testing</a><br>
[2] <a href="https://github.com/KeepSafe/Switchboard" rel="noreferrer" target="_blank">https://github.com/KeepSafe/Switchboard</a><br>
[3]<br>
<a href="http://keepsafe-engineering.tumblr.com/post/28437940369/easy-mobile-ab-testing" rel="noreferrer" target="_blank">http://keepsafe-engineering.tumblr.com/post/28437940369/easy-mobile-ab-testing</a><br>
[4] <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1196897" rel="noreferrer" target="_blank">https://bugzilla.mozilla.org/show_bug.cgi?id=1196897</a><br>
[5] <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1199859" rel="noreferrer" target="_blank">https://bugzilla.mozilla.org/show_bug.cgi?id=1199859</a><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://mail.mozilla.org/pipermail/mobile-firefox-dev/attachments/20150903/2c784c85/attachment-0001.html" rel="noreferrer" target="_blank">http://mail.mozilla.org/pipermail/mobile-firefox-dev/attachments/20150903/2c784c85/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Thu, 3 Sep 2015 11:25:07 -0700<br>
From: Martin Thomson <<a href="mailto:mt@mozilla.com">mt@mozilla.com</a>><br>
To: Mark Finkle <<a href="mailto:mfinkle@mozilla.com">mfinkle@mozilla.com</a>><br>
Cc: "group, mozilla.dev.planning" <<a href="mailto:dev-planning@lists.mozilla.org">dev-planning@lists.mozilla.org</a>>,<br>
"<a href="mailto:mobile-firefox-dev@mozilla.org">mobile-firefox-dev@mozilla.org</a>" <<a href="mailto:mobile-firefox-dev@mozilla.org">mobile-firefox-dev@mozilla.org</a>><br>
Subject: Re: A/B Testing in Firefox for Android<br>
Message-ID:<br>
<CAPLxc=<a href="mailto:UKa9JF%2BWVjYTRPXfe-8_kuopNUH9xZE4oZ5BJASoop8Q@mail.gmail.com">UKa9JF+WVjYTRPXfe-8_kuopNUH9xZE4oZ5BJASoop8Q@mail.gmail.com</a>><br>
Content-Type: text/plain; charset=UTF-8<br>
<br>
This seems like a good idea.<br>
<br>
Has anyone considered contributing a change to switchboard that would<br>
allow the experiment policy to be downloaded to clients and the A/B<br>
decision made there? That removes any possibility of creating another<br>
potential tracking mechanism.<br>
<br>
On Thu, Sep 3, 2015 at 11:16 AM, Mark Finkle <<a href="mailto:mfinkle@mozilla.com">mfinkle@mozilla.com</a>> wrote:<br>
> We have decided to start running A/B Testing [1] in Firefox for Android.<br>
> These experiments are intended to optimize specific outcomes, as well as,<br>
> inform our long-term design decisions. We want to create the best Firefox<br>
> experience we can, and these experiments will help.<br>
><br>
> The system will also allow us to throttle the release of features, called<br>
> staged rollout, so we can monitor new features in a controlled manner<br>
> across a large userbase and a fragmented device ecosystem. If we need to<br>
> rollback a feature for some reason, we'd have the ability to do that.<br>
><br>
> Technical details:<br>
> * Switchboard is used to control experiment segmenting and staged rollout.<br>
> * Telemetry is used to collect metrics about an experiment.<br>
> * FHR is used to track active experiments so we can correlate to<br>
> application usage.<br>
><br>
> == What is Switchboard? ==<br>
><br>
> Switchboard is an open source SDK for doing A/B testing and staged rollouts<br>
> [2][3]. It connects to a server component, which maintains a list of active<br>
> experiments.<br>
><br>
> The SDK does create a UUID, which is stored on the device. The UUID is sent<br>
> to the server, which uses it to "bucket" the client, but the UUID is never<br>
> stored on the server. In fact, the server does not store any data. The<br>
> server we are using is being hosted by Mozilla.<br>
><br>
> We decided to start using Switchboard because it's simple, open source,<br>
> saves no data and can be hosted by Mozilla. Thanks to the KeepSafe folks<br>
> for releasing Switchboard.<br>
><br>
> == Planning Experiments ==<br>
><br>
> The Mobile Product and UX teams are the primary drivers for creating<br>
> experiments, but as is common on the Mobile team, ideas can come from<br>
> anywhere. We have been working with the Mozilla Growth team, getting a<br>
> better understanding of how to design the experiments and analyze the<br>
> metrics. UX researchers will also have input into the experiments.<br>
><br>
> Once Product and UX complete the experiment design, Development would land<br>
> code in Firefox to implement the desired variations of the experiment.<br>
> Development would also land code in the Switchboard server to control the<br>
> configuration of the experiment: Is it active? How are the variations<br>
> distributed across the user population?<br>
><br>
> Since we use Telemetry to collect metrics on the experiments, the Beta<br>
> channel is likely our best time period to run experiments. Telemetry is on<br>
> by default on Nightly, Aurora and Beta; and Beta is the largest userbase of<br>
> those three channels.<br>
><br>
> Once we decide which variation of the experiment is the "winner", we'll<br>
> change the Switchboard server configuration for the experiment so that 100%<br>
> of the userbase will flow through the winning variation.<br>
><br>
> Yes, a small percentage of the Release channel has Telemetry enabled, but<br>
> it might be too small to be useful for experimentation. Time will tell.<br>
><br>
> Note: Switchboard itself will be enabled on all channels. It collects no<br>
> data and gives us a "code-free" way of staging rollouts. It much less risky<br>
> and time consuming than uplifting patches that need to land on branches at<br>
> specific times.<br>
><br>
> == What's Happening Now? ==<br>
><br>
> Switchboard has already landed in Nightly [4] and is currently behind a<br>
> Nightly build flag. Once we feel comfortable, we'll let it ride the trains.<br>
> Our first experiment will likely be testing a new onboarding experience [5].<br>
><br>
> [1] <a href="https://en.wikipedia.org/wiki/A/B_testing" rel="noreferrer" target="_blank">https://en.wikipedia.org/wiki/A/B_testing</a><br>
> [2] <a href="https://github.com/KeepSafe/Switchboard" rel="noreferrer" target="_blank">https://github.com/KeepSafe/Switchboard</a><br>
> [3]<br>
> <a href="http://keepsafe-engineering.tumblr.com/post/28437940369/easy-mobile-ab-testing" rel="noreferrer" target="_blank">http://keepsafe-engineering.tumblr.com/post/28437940369/easy-mobile-ab-testing</a><br>
> [4] <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1196897" rel="noreferrer" target="_blank">https://bugzilla.mozilla.org/show_bug.cgi?id=1196897</a><br>
> [5] <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1199859" rel="noreferrer" target="_blank">https://bugzilla.mozilla.org/show_bug.cgi?id=1199859</a><br>
> _______________________________________________<br>
> dev-planning mailing list<br>
> <a href="mailto:dev-planning@lists.mozilla.org">dev-planning@lists.mozilla.org</a><br>
> <a href="https://lists.mozilla.org/listinfo/dev-planning" rel="noreferrer" target="_blank">https://lists.mozilla.org/listinfo/dev-planning</a><br>
<br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
mobile-firefox-dev mailing list<br>
<a href="mailto:mobile-firefox-dev@mozilla.org">mobile-firefox-dev@mozilla.org</a><br>
<a href="https://mail.mozilla.org/listinfo/mobile-firefox-dev" rel="noreferrer" target="_blank">https://mail.mozilla.org/listinfo/mobile-firefox-dev</a><br>
<br>
<br>
------------------------------<br>
<br>
End of mobile-firefox-dev Digest, Vol 31, Issue 4<br>
*************************************************<br>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><b>Anthony Lam</b><br><span>Senior Product Designer – Firefox for Android <br></span></div><div><span><a href="mailto:alam@mozilla.com" target="_blank">alam@mozilla.com</a><br></span><span style="color:rgb(41,47,51);font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-size:14px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:20px;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none;background-color:rgb(255,255,255)"></span></div></div></div></div></div></div></div>
</div>