[activity-stream] Decision/Update on Status of Activity Stream on FX57

Ritu Kothari rkothari at mozilla.com
Wed Nov 8 00:17:34 UTC 2017


On Tue, Nov 7, 2017 at 11:58 AM, Tim Spurway <tspurway at mozilla.com> wrote:

> I can clarify this one, Jenn!
>
> Hi Ritu,
>
> We will be preffing AS ON for 100% of the population in release 57 Desktop
> by default.  100% rollout.
>
> We will run two studies, starting immediately after release:
>
> 1)  1% Global Fx Desktop ALL USERS pref off AS to study general affect of
> AS on overall retention
> 2)  1% Global NEW USERS pref off AS to study how AS affects new user
> retention
>

Thanks Tim. This sounds perfect.

>
> cheers,
> tim
>
> On Nov 7, 2017, at 2:40 PM, Ritu Kothari <rkothari at mozilla.com> wrote:
>
>
>
> On Tue, Nov 7, 2017 at 6:03 AM, Jenn Chaulk <jchaulk at mozilla.com> wrote:
>
>> Hello,
>>
>> Since the last update on October 26th, the Activity Stream team has been
>> working to finalize a solid contingency plan for FX57, and also analyzing
>> (with the help of other experts across the org) the data available from
>> past studies, as well as the 57 Beta Shield Study on Activity Stream
>> Retention that is currently in flight.
>>
>> Yesterday the confirmed details of that contingency plan, as well as the
>> findings from the above mentioned data gathering and analysis were
>> presented to management for review.  The presentation can be reviewed here
>> -  Activity Stream: 57 Go/No-Go Decision
>> <https://docs.google.com/presentation/d/1ynPPRRtwALM8zbNxZRTAyoFFFt2UvBZfd_ANVLCEYuc/edit#slide=id.p>
>> .
>>
>> In summary, we had previously seen evidence in 56 Release indicating a
>> drop in retention for Activity Stream users vs the legacy new tab
>> experience.  Further analysis of the 56 data has suggested that these
>> results are actually coming from missing data in the experiment dataset,
>> rather than from an actual drop in retention.  (*Note *- the cause of
>> this missing data is still unclear.) Also, we were unable to reproduce such
>> a retention drop while running a similar study in 57 Beta, indicating that
>> Activity Stream does not actually have any negative impact on user
>> retention in FX57.
>>
>> Upon conclusion of the presentation the recommendation from management
>> was to proceed as follows:
>>
>>    - *Proceed as per original plan: Activity Stream will be rolled out
>>    to 99% of the FX57 population upon release.  A 1% control group will remain
>>    on the legacy experience.*
>>
>> Thanks for your email Jenn. In the past, when we've had a GO decision on
> features and it is a pref controlled feature, we typically go live with the
> pref turned ON from the moment we launch i.e. 100% rollout. In this case,
> are we planning to go-live with the pref for AS set to ON and planning to
> run a SHIELD study that turns AS off for 1% of release population?
>
> Thanks,
> Ritu
>
> The team will continue to meet regularly and review early data from
>> release, to monitor for any issues or abnormalities.  If any major issues
>> do occur the team may adjust the release parameters for FX57 (i.e., pull
>> back to a smaller population percentage) as the situation dictates.
>>
>> Best Regards,
>> Jenn
>>
>> --
>>
>> Jennifer A. Chaulk - PMP
>> Engineering Program Manager
>> Mozilla Corporation
>> Work: 1.416.848.3114 x 3029 <(416)%20848-3114>
>> Mobile: 519.277.0365 <(519)%20277-0365>
>> Email: jchaulk at mozilla.com
>> IRC: jchaulk
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "epm-status" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to epm-status+unsubscribe at mozilla.com.
>> To post to this group, send email to epm-status at mozilla.com.
>> To view this discussion on the web visit https://groups.google.com/a/mo
>> zilla.com/d/msgid/epm-status/CAOxuPHExAVpnGKXfbNWrRJdMGoJe6C
>> UYiev04tB8Bcp8Lv%3DSrw%40mail.gmail.com
>> <https://groups.google.com/a/mozilla.com/d/msgid/epm-status/CAOxuPHExAVpnGKXfbNWrRJdMGoJe6CUYiev04tB8Bcp8Lv%3DSrw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Activity Stream Team" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to activity-stream+unsubscribe at mozilla.com.
> To post to this group, send email to activity-stream at mozilla.com.
> To view this discussion on the web visit https://groups.google.com/a/
> mozilla.com/d/msgid/activity-stream/CAERYvbY7%
> 2B20S9VZkoWZq4GNep4nh4sMVr4B5zJ0ddvEX-CM6Rw%40mail.gmail.com
> <https://groups.google.com/a/mozilla.com/d/msgid/activity-stream/CAERYvbY7%2B20S9VZkoWZq4GNep4nh4sMVr4B5zJ0ddvEX-CM6Rw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20171107/b38749f1/attachment.html>


More information about the firefox-dev mailing list