<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<br>
<div class="moz-cite-prefix">On 7/31/2015 5:08 PM, Matthew N. wrote:<br>
</div>
<blockquote
cite="mid:CANcCaXe9W8cX7N3sdhuoHUzUrzbewzcN8sVa63DYT+TFJfYG1A@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">Perhaps the
backlog is the place where the upcoming quality teams will
pull from as its ideally identifying important issues in all
of our components. A follow-up question would be if anyone is
pruning this list?<br>
</div>
</div>
</blockquote>
<br>
I do have a plan for how bugzilla fits in with our broader quality
efforts, and this backlog flag is probably not part of it. But I'd
like to go into a little more detail about how I'm approaching the
problem:<br>
<br>
Bug reports are the primary unit of quality assurance in software.
Bug reports from our users, but especially pre-release users, are
essential to building a quality product. We at Mozilla have lived so
long in a world where we don't triage and make a decision about
every bug report that we're numb to the pain that it's causing us.
We need to make some dramatic changes in how we handle bug triage
and the tracking and decision-making process around bugs.<br>
<br>
This past quarter we've started small: trying to get the bugs filed
each day from Firefox:Untriaged into the right bugzilla component.
But the long-term goal is bigger:<br>
<br>
<ul>
<li>Find or promote a triage and bug handling lead for every
bugzilla component. (If no lead can be found, it's probably a
sign that the code should become Dead).<br>
</li>
<li>Make a crisp decision about every bug.</li>
</ul>
<p>I've been thinking a lot about what it means to make a decision
about every bug, and I don't have final answers, but I do think
that we need to end up with the following categories which we can
apply to every bug in bugzilla:<br>
</p>
<ul>
<li>newly filed - next action is component lead<br>
</li>
<li>under investigation - clear next step(s) identified</li>
<li>ready for a product/engineering priority decision - next
action is component lead, engineering lead, or product lead<br>
</li>
<li>Priority decided<br>
</li>
<ul>
<li>must fix by release N - typically tracked by release drivers
or program managers<br>
</li>
<li>work backlog for an engineering team - tracked by component
lead or other team lead<br>
</li>
<li>a valid bug, but not something an engineering team will even
prioritize - not tracked<br>
</li>
</ul>
<li>bug currently being fixed (owner identified)</li>
<li>bug partially fixed (not all QA is done, or not
upstreamed/verified on all the necessary branches) - next-step
engineering or QA owner identified<br>
</li>
<li>bug completely fixed, done - woot!<br>
</li>
<li>otherwise closed: wontfix/incomplete/invalid etc<br>
</li>
</ul>
We're not very far away from this model right now: there's a lot of
fuzz at the beginning of the process and there's a lot of fuzz
around how we express bug decision and priorities across our various
teams. But mostly the problem is that we don't have enough people
triaging bugs, following up on action items, and then make priority
decisions.<br>
<br>
We're working on that in several ways:<br>
<ul>
<li>this week I'm posting a job posting for a bugmaster, which
will be a combination program management/QA role to help keep
track of components and their owners, create reports, and
generally the bugzilla zen garden.</li>
<li>Bug triage will be a core focus of QA activity later in this
year. We are working on a significant QA contract to help with
bug triage and bugzilla maintenance, and are continuing efforts
to expand community involvement in bug triage and related tasks
such as creating testcases, finding regression ranges, etc.<br>
</li>
<li>A key part of "great or dead" will be triaging the entire
backlog of bugs in related bugzilla components as we get to
them. So for instance as we do a great-or-dead review of the
Firefox bookmarks system, we will review all of the open bugs in
"Firefox:Bookmarks & History" and "Toolkit:Places" so that
all the bugs fit into the buckets above at the end of the
project.<br>
</li>
</ul>
Relative to the specific firefox-backlog bug flag, my experience has
been that we don't really use it. We were supposed to pick bugs for
the priority backlog from the larger list of firefox-backlog bugs.
But in practice, tech leads have in their head the list of bugs they
need to accomplish for particular projects, and so the backlog is
mostly not looked at once things have been added to it. I don't have
a stake in whether we keep it or throw it out for now, but
eventually when we have a bugmaster who can spend their full
attention figuring out the best way to organize bugzilla flags
across multiple teams, I'm hoping we can come up with something more
organized and available across multiple teams.<br>
<br>
--BDS<br>
<br>
</body>
</html>