Important bugs list

Joshua Cranmer pidgeot18 at gmail.com
Mon Sep 24 04:46:51 UTC 2012


On 9/21/2012 5:23 PM, Wayne Mery (d531) wrote:
> The QA breakout at mozcamp [1] discussed making a list of high value, 
> i.e. "most important" bugs list for developer attention and action. 
> This is to be a small, selective set of bugs, culled from a larger set 
> of bugs, to serve as a managable "grab bag" for developers.  The idea 
> is, as bugs on the list are fixed they are removed from the list, and 
> other bugs added to the list. ("important bugs" is not an official 
> name - so let's not haggle yet over names)

The working name I've given this is "The List," which, admittedly, isn't 
very descriptive to outsiders.
>
> This is not a list to make everyone happy; not every user will see one 
> of their issues here. The goal is to tackle a focused list, and add to 
> and subtract from it as progress is made.
>
> Per the meeting, there are 4 dimensions to categorizing bugs for this 
> list:
> 1. How many people are likely to be affected
> 2. How serious is the effect when it occurs (workaround? consequences)
>    [severity as defined by bugzilla]
> 3. How much effort to fix this bug?
> 4. How reproducible it is
>
> To summarize what I think is the current situation
>
> #1 is going to be a judgment call by QA, in come cases informed by 
> feedback from support forum - feedback which we currently don't have 
> except the unweighted [gs] whiteboard item
>
> #2 we know from bugzilla - unless the severity setting is incorrect. 
> When you touch a bug you should check the severity and change it if 
> you feel it isn't accurate. Ref: 
> https://bugzilla.mozilla.org/page.cgi?id=fields.html#bug_severity

As a developer and occasional bug filer, I tend to mentally divide the 
severity level in three categories: blocker, for bugs which amount to 
"would close the tree right now but for extenuating circumstances"; 
enhancement, for bugs which describe RFEs; and everything else. And I 
have seen the odd bug where people argue if it's major or normal (or 
sometimes blocker/critical, but those tend to be mostly people who are 
extremely vocal about their issues), so severity always struck me as 
pretty meaningless.
> #3 can be judged by developers, and for some bugs probably by trigers. 
> To what extent it affects whether a bug gets on the list remains to be 
> seen.

The consensus I recall from the QA meeting was that this aspect wouldn't 
play a significant role in deciding admissibility to the list, but if a 
bug stayed on the list for a long time, this could be a deciding factor 
in kicking it off the list.
> #4 well, presumably every confirmed bug is reproducible to some 
> extent. But we know this is not true. If we continue to feel this is 
> an important factor to a bug being on the list, know that it will 
> require *signficant* manpower to accomplish as it will require 
> reasessing a high percentage of our bugs.

One thing I should point out is that there is a sliding scale of 
reproducibility. Reliably reproducible bugs (i.e., this sequence of 
actions will always cause the bug) are ideal, but if a bug has 
particularly negative impact (scores high in points #1 and #2), the bar 
should instead be at least as low as "the bug sometimes happens when you 
do X."
> For my own sanity I excluded large numbers of bugs, like address book 
> and filters because the former will hopefully be revamped in the next 
> year/in our lifetime, and the later because it has already been 
> heavily triaged in recent months due to the efforts of aceman and 
> others. I excluded international, build, security either because I 
> have no experience in those areas or because the area is highly 
> specialized and likely not of interest to the average developer. I 
> excluded severity=minor and severity=normal.

Since normal is the default, and given that severity has in the past 
been indicated to people as being effectively meaningless, I think 
excluding normal bugs from consideration is the wrong thing to do in the 
long term, although I appreciate that you did it for your own sanity.

-- 
Joshua Cranmer
News submodule owner
DXR coauthor




More information about the tb-planning mailing list