Schedule Driven Releases: Channel Strategy
Wayne Mery (vn)
vseerror at Lehigh.EDU
Thu Apr 21 17:26:34 UTC 2011
On 4/21/2011 8:47 AM, Robert Kaiser wrote:
> [actually sending to list - thanks for not using a newsgroup, where
> reply would work properly]
> Mark Banner schrieb:
>> Therefore, I think we should go for a one-to-one mapping of:
>> * Thunderbird trunk/nightly uses Firefox nightly
>> * Thunderbird Experimental (name TBD) uses Firefox Experimental/Aurora
>> * Thunderbird Beta uses Firefox Beta
>> * Thunderbird Release uses Firefox Release
> Do you think you'll have enough users on all those four channels to make
> them test-worthy? Note that we found on Firefox that less than ~100k
> users on a "channel" probably doesn't get useful enough crash stats for
> efficiently attacking stability (which is currently a problem on both
> nightly and aurora), but Thunderbird might care less about that aspect
> and rely on most crashes being core and detected by us on the FF side
The prospects of shorter release cycles, which might not have point
releases, may indeed be something we should be concerned about. However,
we are significantly different from Firefox:
- Thunderbird isn't going to get there until at least summer.
- Thunderbird doesn't anywhere near the percentage of testing population
that Firefox has. Which explains ...
- Since 3.0 released**, crash-stats information for trunk may as well be
called non-existent. So I don't foresee having enough TB nightly users
to be able to shake out crashes as they appear, or before they can
affect the masses. And there is zero correlation of trunk crash rankings
to branch rankings.
- The same is almost true of current release branch "pre" nightlies. But
there is a better correlation in rankings to the actual release.
As for Firefox possibly covering our butts crash wise, I wouldn't count
on it. Firefox and Thunderbird top crash rankings/priorities tend not to
align. Even though roughly 75% of our top 300 crashes are in
In short, I'm not sure fragmenting the testing population would have
much effect from a crash detection POV or in a way that helps improve
stability of the "next" release. However, the dynamics the releases and
distribution of the user population may change significantly as we get
into shorter release cycles, so I'd say anything's possible.
For now, I think it's more important think about the future in terms of
this question - How do we structure the release process and what we put
into releases, so that we attract *more* users to non-release branches,
and thus increase the total pool of testers?
** During the one or two betas in last weeks of run up to 3.0, we had a
large upsurge in users, but even then I don't recall that lots of
crashes were reported, or that the picture on crash-stats changed
More information about the tb-planning