Schedule Driven Releases: Channel Strategy

Wayne Mery (vn) vseerror at Lehigh.EDU
Thu Apr 21 17:26:34 UTC 2011


On 4/21/2011 8:47 AM, Robert Kaiser wrote:
> [actually sending to list - thanks for not using a newsgroup, where
> reply would work properly]
>
> Mark Banner schrieb:
>> Therefore, I think we should go for a one-to-one mapping of:
>>
>> * Thunderbird trunk/nightly uses Firefox nightly
>> * Thunderbird Experimental (name TBD) uses Firefox Experimental/Aurora
>> * Thunderbird Beta uses Firefox Beta
>> * Thunderbird Release uses Firefox Release
>
> Do you think you'll have enough users on all those four channels to make
> them test-worthy? Note that we found on Firefox that less than ~100k
> users on a "channel" probably doesn't get useful enough crash stats for
> efficiently attacking stability (which is currently a problem on both
> nightly and aurora), but Thunderbird might care less about that aspect
> and rely on most crashes being core and detected by us on the FF side
> anyhow.

The prospects of shorter release cycles, which might not have point 
releases, may indeed be something we should be concerned about. However, 
we are significantly different from Firefox:
- Thunderbird isn't going to get there until at least summer.
- Thunderbird doesn't anywhere near the percentage of testing population 
that Firefox has. Which explains ...
- Since 3.0 released**, crash-stats information for trunk may as well be 
called non-existent. So I don't foresee having enough TB nightly users 
to be able to shake out crashes as they appear, or before they can 
affect the masses. And there is zero correlation of trunk crash rankings 
to branch rankings.
- The same is almost true of current release branch "pre" nightlies. But 
there is a better correlation in rankings to the actual release.

As for Firefox possibly covering our butts crash wise, I wouldn't count 
on it. Firefox and Thunderbird top crash rankings/priorities tend not to 
align. Even though roughly 75% of our top 300 crashes are in 
toolkit/core/nspr-land.

In short, I'm not sure fragmenting the testing population would have 
much effect from a crash detection POV or in a way that helps improve 
stability of the "next" release. However, the dynamics the releases and 
distribution of the user population may change significantly as we get 
into shorter release cycles, so I'd say anything's possible.

For now, I think it's more important think about the future in terms of 
this question - How do we structure the release process and what we put 
into releases, so that we attract *more* users to non-release branches, 
and thus increase the total pool of testers?


** During the one or two betas in last weeks of run up to 3.0, we had a 
large upsurge in users, but even then I don't recall that lots of 
crashes were reported, or that the picture on crash-stats changed 
significantly.



More information about the tb-planning mailing list