<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
For those interested in new quality efforts, I recommend checking
out Mike Hoye's awesome TriageBot effort. He is crowd-sourcing new
bug triage by emailing volunteers one untriaged bug a day. In just
three weeks, 120 contributors have reversed the tide from 120–160
new untriaged bugs per day to just five!<br>
<br>
<a class="moz-txt-link-freetext" href="https://triagebot.mozilla.org/">https://triagebot.mozilla.org/</a><br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 8/12/15 2:16 PM, Benjamin Smedberg
wrote:<br>
</div>
<blockquote cite="mid:55CBB7AA.5040608@smedbergs.us" type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
<div class="moz-cite-prefix">On 7/31/2015 5:08 PM, Matthew N.
wrote:<br>
</div>
<blockquote
cite="mid:CANcCaXe9W8cX7N3sdhuoHUzUrzbewzcN8sVa63DYT+TFJfYG1A@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">Perhaps the
backlog is the place where the upcoming quality teams will
pull from as its ideally identifying important issues in
all of our components. A follow-up question would be if
anyone is pruning this list?<br>
</div>
</div>
</blockquote>
<br>
I do have a plan for how bugzilla fits in with our broader quality
efforts, and this backlog flag is probably not part of it. But I'd
like to go into a little more detail about how I'm approaching the
problem:<br>
<br>
Bug reports are the primary unit of quality assurance in software.
Bug reports from our users, but especially pre-release users, are
essential to building a quality product. We at Mozilla have lived
so long in a world where we don't triage and make a decision about
every bug report that we're numb to the pain that it's causing us.
We need to make some dramatic changes in how we handle bug triage
and the tracking and decision-making process around bugs.<br>
<br>
This past quarter we've started small: trying to get the bugs
filed each day from Firefox:Untriaged into the right bugzilla
component. But the long-term goal is bigger:<br>
<br>
<ul>
<li>Find or promote a triage and bug handling lead for every
bugzilla component. (If no lead can be found, it's probably a
sign that the code should become Dead).<br>
</li>
<li>Make a crisp decision about every bug.</li>
</ul>
<p>I've been thinking a lot about what it means to make a decision
about every bug, and I don't have final answers, but I do think
that we need to end up with the following categories which we
can apply to every bug in bugzilla:<br>
</p>
<ul>
<li>newly filed - next action is component lead<br>
</li>
<li>under investigation - clear next step(s) identified</li>
<li>ready for a product/engineering priority decision - next
action is component lead, engineering lead, or product lead<br>
</li>
<li>Priority decided<br>
</li>
<ul>
<li>must fix by release N - typically tracked by release
drivers or program managers<br>
</li>
<li>work backlog for an engineering team - tracked by
component lead or other team lead<br>
</li>
<li>a valid bug, but not something an engineering team will
even prioritize - not tracked<br>
</li>
</ul>
<li>bug currently being fixed (owner identified)</li>
<li>bug partially fixed (not all QA is done, or not
upstreamed/verified on all the necessary branches) - next-step
engineering or QA owner identified<br>
</li>
<li>bug completely fixed, done - woot!<br>
</li>
<li>otherwise closed: wontfix/incomplete/invalid etc<br>
</li>
</ul>
We're not very far away from this model right now: there's a lot
of fuzz at the beginning of the process and there's a lot of fuzz
around how we express bug decision and priorities across our
various teams. But mostly the problem is that we don't have enough
people triaging bugs, following up on action items, and then make
priority decisions.<br>
<br>
We're working on that in several ways:<br>
<ul>
<li>this week I'm posting a job posting for a bugmaster, which
will be a combination program management/QA role to help keep
track of components and their owners, create reports, and
generally the bugzilla zen garden.</li>
<li>Bug triage will be a core focus of QA activity later in this
year. We are working on a significant QA contract to help with
bug triage and bugzilla maintenance, and are continuing
efforts to expand community involvement in bug triage and
related tasks such as creating testcases, finding regression
ranges, etc.<br>
</li>
<li>A key part of "great or dead" will be triaging the entire
backlog of bugs in related bugzilla components as we get to
them. So for instance as we do a great-or-dead review of the
Firefox bookmarks system, we will review all of the open bugs
in "Firefox:Bookmarks & History" and "Toolkit:Places" so
that all the bugs fit into the buckets above at the end of the
project.<br>
</li>
</ul>
Relative to the specific firefox-backlog bug flag, my experience
has been that we don't really use it. We were supposed to pick
bugs for the priority backlog from the larger list of
firefox-backlog bugs. But in practice, tech leads have in their
head the list of bugs they need to accomplish for particular
projects, and so the backlog is mostly not looked at once things
have been added to it. I don't have a stake in whether we keep it
or throw it out for now, but eventually when we have a bugmaster
who can spend their full attention figuring out the best way to
organize bugzilla flags across multiple teams, I'm hoping we can
come up with something more organized and available across
multiple teams.<br>
<br>
--BDS<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
firefox-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:firefox-dev@mozilla.org">firefox-dev@mozilla.org</a>
<a class="moz-txt-link-freetext" href="https://mail.mozilla.org/listinfo/firefox-dev">https://mail.mozilla.org/listinfo/firefox-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>