<div dir="ltr"><div>Coming back on this thread. I am sure that Gijs and others will think about it but even with the approval-by-default, we still need the status flags (<i>affected</i> => <i>fixed/verified</i>) to be updated once the patch reaches aurora.<br></div>Release management relies on the values of the status flags.<br><br></div><div class="gmail_extra"><br><div class="gmail_quote">2015-12-10 20:43 GMT+01:00 Sylvestre Ledru <span dir="ltr"><<a href="mailto:sylvestre@mozilla.com" target="_blank">sylvestre@mozilla.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span class="">
Hello,<br>
<br>
First, I understand your pain and frustration. We have plan to
simplify the aurora/devedition uplift processes for next year.<br>
<br>
I think we can give a try to your proposition in 46. However, we
think we will need two things:<br>
* usage of a a=css-image-only approval<br>
* a hook to verify the list of files being touched in the commit.<br>
<br>
I reported this bug:<br>
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1231768" target="_blank">https://bugzilla.mozilla.org/show_bug.cgi?id=1231768</a><br></span>
to get this hook.<br>
<br>
Thanks for this proposition,<span class=""><br>
Sylvestre<br>
<br>
<br>
<br>
<div>Le 27/11/2015 11:21, Lawrence Mandel a
écrit :<br>
</div>
</span><div><div class="h5"><blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>Good discussion. A few thoughts on top and in
summary of what's already been said:<br>
<br>
</div>
- I think experimenting is good. 46 seems like a good
time in that we have some time to work out all of the
details.<br>
</div>
- The relman team agrees that while we do get value from
Aurora approvals there is a lot of overhead here - and I
would say ideally there are better activities on which to
spend our time.<br>
</div>
<div>- Much of the value that relman gets out of approvals
should be obtainable from other sources. We should
identify the real value (I think we can do this quickly)
and figure out how to get this information from the
pushlog and other sources.<br>
</div>
- There is an issue of trust. While many (in my experience
the majority) of Mozilla devs have demonstrated that they
make good decisions wrt uplifts, some don't. Having an open
m-a branch is scary. We really need to keep this branch
stable. This is the one piece of value, control over
landings, that will be hard to replace. I agree with
experimenting as we don't know how much actual value we're
getting with our current model.<br>
<br>
</div>
Sylvestre - Can we please put this on our list to discuss in
Orlando and come back with either a commitment to do what Gijs
has proposed or an alternative proposal?<br>
<br>
</div>
Lawrence<br>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Nov 26,
2015 at 7:28 AM, Gijs Kruitbosch <span dir="ltr"><<a href="mailto:gijskruitbosch@gmail.com" target="_blank"></a><a href="mailto:gijskruitbosch@gmail.com" target="_blank">gijskruitbosch@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Time for a
recap now this discussion has died
down a little.<br>
<br>
TL;DR: I would like to experiment with
this for the 46 cycle to see how this
works in practice. If we find it has
negative effects, we can stop doing it
(immediately; we don't need to wait
for the end of the cycle...). If we
find it has broadly positive effects
(ie doesn't lead to more breakage than
usual), we can look at broadening the
set of things we uplift without going
through the formal approval process we
currently use.<span><br>
<br>
On 20/11/2015 19:22, Gijs Kruitbosch
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I would like to propose that we
eliminate the explicit aurora
approval requirement for
changesets that meet ALL of the
following criteria:<br>
- only change CSS and/or image
files<br>
- are landing in the first 4 weeks
of the aurora cycle;<br>
- have landed on m-c and stuck<br>
</blockquote>
<br>
</span><span>
On 20/11/2015 19:33, Robert Kaiser
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
How many of those do we have every
cycle (i.e. is it enough that it
gives us a significant benefit and
even warrants discussion)?<br>
</blockquote>
</span>
I don't know. I tried to find out, and
it's hard to get that information. I'm
fairly sure it's a comparatively small
(<10%) subset of the patches that
gets uplift, though.<br>
<br>
I think it warrants discussion because
I think there is broad agreement from
relmgmt, QE and engineers that the
current status quo is not ideal, but
less agreement on what we should be
aiming for instead. I'd like us to
experiment and evolve our strategy
here. This is not me personally going
"ugh, I wish the patches I wrote
didn't have to do this approval
dance". This is me going "I wonder if
this is a good set of things to use to
evolve how we deal with approvals"
(though full disclosure, because of
the type of patches I often write,
this probably does benefit me
personally more than it will some
other people). I'll get back to this
point replying to some of the other
posts further downthread.<span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
What's the definition of "have
stuck"? For how long?<br>
</blockquote>
</span>
"landed on m-c without being backed
out". I don't really want to use extra
criteria like "landed and stuck for X
days" because I think the criteria are
already pretty narrow, and adding
verification burden here is not going
to help materially.<span><br>
<br>
On 20/11/2015 19:34, Chris Peterson
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Why not allow any code change to
be uplifted to Aurora based on the
developer's discretion? Or allow a
patch's reviewer to a+ for Aurora
uplift?<br>
</blockquote>
</span>
I think "developer's discretion" alone
is not likely to be conducive to
stability. I think patch reviewer
and/or third person review by another
person (not necessarily relman) might
be interesting to explore, but still
more high-risk than what I am
suggesting. As an experiment, I'd like
to start with what I originally
suggested. We can evaluate after 1
cycle (or before if it blows up in our
face).<span><br>
<br>
On 20/11/2015 19:55, Liz Henry
(:lizzard) wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I could be ok with us outlining
this kind of criteria and letting
developers know they can land
their own changes in some cases.
Verifying the fix on m-c (by
someone other than the developer)
might help too.<br>
<br>
But, what those cases are, I'm not
entirely sure. I think there have
been css only uplifts that cause
significant regressions. Now, me
reviewing them isn't going to
catch those regressions. But my
being aware of a cluster of uplift
requests can mean I know to ask QE
for extra testing in particular
areas. I factor the uplift action
into a more holistic view of
what's happening across the board
for a release and the amount of
change and risk we're looking at.
That isn't always visible to the
developers requesting uplift.<br>
<br>
Gijs you mention devs not being
"required" to do this. I wonder if
this process might normalize
uplifting things that could just
as well ride the trains.<br>
<br>
The value of asking people to ask
for uplift is sometimes in that
they take their changes more
seriously. Often, people talk to
me first on irc before even
asking, and during that
conversation they find more work
to do, they think about testing or
verifying fixes, or I uncover more
uplifts that need to happen for
their work to land on aurora.<br>
</blockquote>
<br>
</span>
I think all of these paragraphs point
to a single (valid!) concern:
uplifting things increases
instability. We can contain the risk
of instability by not uplifting
things, or at least by asking for
automated tests and/or manual QE, or
restricting what things we uplift
(interface changes, patch size, l10n
concerns, etc. (l10n being a bit
special because there are other
reasons besides stability)).<br>
<br>
But as dbaron and past have pointed
out (and as I've discussed with Liz
privately), realistically part of the
issue is that we often don't get QE
verification or crash-rate-combatting
until we get to beta/release. That
means nightly and, to a slightly
lesser degree, aurora, are both not
particularly stable, and that that
stability is not completely dependent
on how strict we are with uplifts.<br>
<br>
It's very hard to assess the success
of the current system at increasing
stability because there is no way to
know "what would have been". We don't
know if developers or relman are
overly cautious or not cautious/strict
enough, unless we experiment. Because
nobody seems particularly happy with
the current state of things, I would
like us to experiment. :-)<br>
<br>
<br>
So perhaps my original phrasing was
clumsy. I would like us to experiment
with being "more permissive" about
uplifts, and I'm purposefully starting
out with a subset of patches that I
suspect are lower risk, and where
there is a clear secondary benefit (ie
devedition theming).<br>
<br>
I believe it's probably equally useful
to take a hard look at improving
quality on nightly and aurora through
other means. I'll open a separate
thread about that.<span><font color="#888888"><br>
<br>
~ Gijs<br>
</font></span></blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>