Decision/Update on Status of Activity Stream on FX57

Ritu Kothari rkothari at
Tue Nov 7 19:40:06 UTC 2017

On Tue, Nov 7, 2017 at 6:03 AM, Jenn Chaulk <jchaulk at> wrote:

> Hello,
> Since the last update on October 26th, the Activity Stream team has been
> working to finalize a solid contingency plan for FX57, and also analyzing
> (with the help of other experts across the org) the data available from
> past studies, as well as the 57 Beta Shield Study on Activity Stream
> Retention that is currently in flight.
> Yesterday the confirmed details of that contingency plan, as well as the
> findings from the above mentioned data gathering and analysis were
> presented to management for review.  The presentation can be reviewed here
> -  Activity Stream: 57 Go/No-Go Decision
> <>
> .
> In summary, we had previously seen evidence in 56 Release indicating a
> drop in retention for Activity Stream users vs the legacy new tab
> experience.  Further analysis of the 56 data has suggested that these
> results are actually coming from missing data in the experiment dataset,
> rather than from an actual drop in retention.  (*Note *- the cause of
> this missing data is still unclear.) Also, we were unable to reproduce such
> a retention drop while running a similar study in 57 Beta, indicating that
> Activity Stream does not actually have any negative impact on user
> retention in FX57.
> Upon conclusion of the presentation the recommendation from management was
> to proceed as follows:
>    - *Proceed as per original plan: Activity Stream will be rolled out to
>    99% of the FX57 population upon release.  A 1% control group will remain on
>    the legacy experience.*
> Thanks for your email Jenn. In the past, when we've had a GO decision on
features and it is a pref controlled feature, we typically go live with the
pref turned ON from the moment we launch i.e. 100% rollout. In this case,
are we planning to go-live with the pref for AS set to ON and planning to
run a SHIELD study that turns AS off for 1% of release population?


The team will continue to meet regularly and review early data from
> release, to monitor for any issues or abnormalities.  If any major issues
> do occur the team may adjust the release parameters for FX57 (i.e., pull
> back to a smaller population percentage) as the situation dictates.
> Best Regards,
> Jenn
> --
> Jennifer A. Chaulk - PMP
> Engineering Program Manager
> Mozilla Corporation
> Work: 1.416.848.3114 x 3029 <(416)%20848-3114>
> Mobile: 519.277.0365 <(519)%20277-0365>
> Email: jchaulk at
> IRC: jchaulk
> --
> You received this message because you are subscribed to the Google Groups
> "epm-status" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to epm-status+unsubscribe at
> To post to this group, send email to epm-status at
> To view this discussion on the web visit
> <>
> .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the firefox-dev mailing list