[activity-stream] Decision/Update on Status of Activity Stream on FX57

Tim Spurway tspurway at mozilla.com
Tue Nov 7 19:58:24 UTC 2017


I can clarify this one, Jenn!

Hi Ritu, 

We will be preffing AS ON for 100% of the population in release 57 Desktop by default.  100% rollout.

We will run two studies, starting immediately after release:

1)  1% Global Fx Desktop ALL USERS pref off AS to study general affect of AS on overall retention
2)  1% Global NEW USERS pref off AS to study how AS affects new user retention

cheers,
tim

> On Nov 7, 2017, at 2:40 PM, Ritu Kothari <rkothari at mozilla.com> wrote:
> 
> 
> 
> On Tue, Nov 7, 2017 at 6:03 AM, Jenn Chaulk <jchaulk at mozilla.com <mailto:jchaulk at mozilla.com>> wrote:
> Hello,
> 
> Since the last update on October 26th, the Activity Stream team has been working to finalize a solid contingency plan for FX57, and also analyzing (with the help of other experts across the org) the data available from past studies, as well as the 57 Beta Shield Study on Activity Stream Retention that is currently in flight.  
> 
> Yesterday the confirmed details of that contingency plan, as well as the findings from the above mentioned data gathering and analysis were presented to management for review.  The presentation can be reviewed here -  Activity Stream: 57 Go/No-Go Decision <https://docs.google.com/presentation/d/1ynPPRRtwALM8zbNxZRTAyoFFFt2UvBZfd_ANVLCEYuc/edit#slide=id.p>.
> 
> In summary, we had previously seen evidence in 56 Release indicating a drop in retention for Activity Stream users vs the legacy new tab experience.  Further analysis of the 56 data has suggested that these results are actually coming from missing data in the experiment dataset, rather than from an actual drop in retention.  (Note - the cause of this missing data is still unclear.) Also, we were unable to reproduce such a retention drop while running a similar study in 57 Beta, indicating that Activity Stream does not actually have any negative impact on user retention in FX57.
> 
> Upon conclusion of the presentation the recommendation from management was to proceed as follows:
> Proceed as per original plan: Activity Stream will be rolled out to 99% of the FX57 population upon release.  A 1% control group will remain on the legacy experience.
> Thanks for your email Jenn. In the past, when we've had a GO decision on features and it is a pref controlled feature, we typically go live with the pref turned ON from the moment we launch i.e. 100% rollout. In this case, are we planning to go-live with the pref for AS set to ON and planning to run a SHIELD study that turns AS off for 1% of release population?
> 
> Thanks,
> Ritu
> 
> The team will continue to meet regularly and review early data from release, to monitor for any issues or abnormalities.  If any major issues do occur the team may adjust the release parameters for FX57 (i.e., pull back to a smaller population percentage) as the situation dictates.
> 
> Best Regards,
> Jenn
> 
> -- 
> 
> Jennifer A. Chaulk - PMP
> Engineering Program Manager
> Mozilla Corporation
> Work: 1.416.848.3114 x 3029 <tel:(416)%20848-3114>
> Mobile: 519.277.0365 <tel:(519)%20277-0365>
> Email: jchaulk at mozilla.com <mailto:jchaulk at mozilla.com>
> IRC: jchaulk
> 
> -- 
> You received this message because you are subscribed to the Google Groups "epm-status" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to epm-status+unsubscribe at mozilla.com <mailto:epm-status+unsubscribe at mozilla.com>.
> To post to this group, send email to epm-status at mozilla.com <mailto:epm-status at mozilla.com>.
> To view this discussion on the web visit https://groups.google.com/a/mozilla.com/d/msgid/epm-status/CAOxuPHExAVpnGKXfbNWrRJdMGoJe6CUYiev04tB8Bcp8Lv%3DSrw%40mail.gmail.com <https://groups.google.com/a/mozilla.com/d/msgid/epm-status/CAOxuPHExAVpnGKXfbNWrRJdMGoJe6CUYiev04tB8Bcp8Lv%3DSrw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "Activity Stream Team" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to activity-stream+unsubscribe at mozilla.com <mailto:activity-stream+unsubscribe at mozilla.com>.
> To post to this group, send email to activity-stream at mozilla.com <mailto:activity-stream at mozilla.com>.
> To view this discussion on the web visit https://groups.google.com/a/mozilla.com/d/msgid/activity-stream/CAERYvbY7%2B20S9VZkoWZq4GNep4nh4sMVr4B5zJ0ddvEX-CM6Rw%40mail.gmail.com <https://groups.google.com/a/mozilla.com/d/msgid/activity-stream/CAERYvbY7%2B20S9VZkoWZq4GNep4nh4sMVr4B5zJ0ddvEX-CM6Rw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20171107/ace7b53d/attachment-0001.html>


More information about the firefox-dev mailing list