[Go Faster] Planning for deployments involving system add-ons or updates to system add-ons
andrei.vaida at softvisioninc.eu
Thu Mar 30 14:32:42 UTC 2017
@Cory a ballpark schedule at least for larger or more complex (as in
requiring a high amount of manual testing) system add-ons would be
great. I think having two rounds of testing (once for the * .xpi file
and once for the release-sysaddon update channel) is a good thing, two
QA filters is better than one and we all want to ship any system add-on
in its best shape possible -- this is something that I think we should
I'm aware that the whole point of Go Faster is to ship
features/fixes/updates as fast as possible, that's why I was rather
thinking of a raw and always-changing calendar that my team could check,
even on a daily-basis. Using the Go Faster Public Calendar
sounds good to me.
@Ben QE is usually notified when the system add-on is ready to be
tested. Automating parts of this process would definitely help.
Overall, this is what we're covering for each system add-on through
that the update.xml file associated to the builds we're covering are
correctthat the system add-on or system add-on update is successfully
installed on the update channelthat the features associated to that
system add-on work as intendedbased on the complexity of the system
add-on in question, this can get quite laboriousthat the other system
add-ons (usually just Pocket) are not affected in an unintended way by
the system add-onthis usually consists of a spot-check on Pocket core
functionality only, but if we need to pay attention to other system
add-ons or updates that are coincidentally deployed around the same
time, then this too can get laboriousWe're doing this on 4 platforms (2
x Windows, 1 x Mac, 1 x Linux), 2 Firefox build architectures (32-bit,
64-bit), 3 locales (en-US, de, ru) and we're checking both the Firefox
build versions that should get the system add-on and a few build
versions that should not get it. Here's an example: 1325872
Thank you both for following up on this so quickly!
------ Original Message ------
From: "Ben Hearsum" <bhearsum at mozilla.com>
To: "Cory Price" <cprice at mozilla.com>
Cc: "Andrei Vaida" <andrei.vaida at softvisioninc.eu>; "Ritu Kothari"
<rkothari at mozilla.com>; "Jason Thomas" <jthomas at mozilla.com>; "Florin
Mezei" <florin.mezei at softvisioninc.eu>; "cornel.ionce at softvisioninc.eu"
<cornel.ionce at softvisioninc.eu>; "Robert Helmer" <rhelmer at mozilla.com>;
"Felipe Gomes" <fgomes at mozilla.com>; gofaster at mozilla.org
Sent: 2017-03-30 4:11:42 PM
Subject: Re: [Go Faster] Planning for deployments involving system
add-ons or updates to system add-ons
>On Wed, Mar 29, 2017 at 03:50:45PM -0700, Cory Price wrote:
>> + some other folks because this is a great topic, so thanks for
>> On Wed, Mar 29, 2017 at 2:14 AM, Andrei Vaida
>><andrei.vaida at softvisioninc.eu
>> > wrote:
>> > Hi Cory,
>> > I was wondering if the Go Faster team has some sort of schedule
>> > for the system add-ons or system add-on updates they're planning to
>> > The process we set up last year for test channel verification
>> > been working really well so far, but I think my team could be more
>> > efficient if there were a release schedule we could access, to
>> > upcoming deployments in advance. It wouldn't have to be a very
>> > schedule, just enough so we know that one or more system add-ons
>> > planned to be deployed "sometime in the week of...".
>> > Such a schedule would help us a lot, especially since there were
>> > cases when my team had to fit in this kind of testing in
>> > days, without knowing and planning for them before hand.
>> > What do you think? Does Go Faster already have such a schedule? And
>> > not, would it be possible to build one and share it with our team?
>> There currently is not such a schedule. The premise of the Go Faster
>> workflow is to turn things around quickly, with the ability to land
>> priority fixes and feature updates in short order.
>I'm coming at this without deep knowledge of this part of the process,
>but I'm curious when QE currently gets looped in? Are they notified as
>soon as we know there's a new System Addon coming, or only when it's
>ready to be tested?
>> For some larger full feature system add-ons such as bug 1351424, or
>> 1351454 building a schedule is reasonable, but the majority of system
>> add-ons we release (at least recently) have been things we have to
>> right away.
>> I don't know what the best way to handle this is. One thing I've been
>> thinking of was revisiting the requirement of two QA verification
>> the process, maybe asking for just one round of QA verification?
>Depending on how deep the "verification of test channel" is, this might
>be something that can be automated. We have a couple of existing
>frameworks that do automated app-update verification. Specifically, if
>these are just "ensure system addon gets installed" (explicitly no
>functional verification of the addon), it might be possible to adapt
>them or do something similar for System Addons. This might be worth
>bringing up with Benson's team, since they're already working on
>automated pipeline for System Addon updates.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Gofaster