Important bugs list

Wayne Mery (d531) vseerror at Lehigh.EDU
Fri Sep 21 22:23:29 UTC 2012

The QA breakout at mozcamp [1] discussed making a list of high value, 
i.e. "most important" bugs list for developer attention and action. This 
is to be a small, selective set of bugs, culled from a larger set of 
bugs, to serve as a managable "grab bag" for developers.  The idea is, 
as bugs on the list are fixed they are removed from the list, and other 
bugs added to the list.  ("important bugs" is not an official name - so 
let's not haggle yet over names)

This is not a list to make everyone happy; not every user will see one 
of their issues here. The goal is to tackle a focused list, and add to 
and subtract from it as progress is made.

Per the meeting, there are 4 dimensions to categorizing bugs for this list:
1. How many people are likely to be affected
2. How serious is the effect when it occurs (workaround? consequences)
    [severity as defined by bugzilla]
3. How much effort to fix this bug?
4. How reproducible it is

To summarize what I think is the current situation

#1 is going to be a judgment call by QA, in come cases informed by 
feedback from support forum - feedback which we currently don't have 
except the unweighted [gs] whiteboard item

#2 we know from bugzilla - unless the severity setting is incorrect. 
When you touch a bug you should check the severity and change it if you 
feel it isn't accurate. Ref:

#3 can be judged by developers, and for some bugs probably by trigers. 
To what extent it affects whether a bug gets on the list remains to be seen.

#4 well, presumably every confirmed bug is reproducible to some extent. 
But we know this is not true. If we continue to feel this is an 
important factor to a bug being on the list, know that it will require 
*signficant* manpower to accomplish as it will require reasessing a high 
percentage of our bugs.

It will be weeks I think until we are in a position to create a list 
based on the above 4 elements and a process to go with it. And even then 
it will likely be iterative - starting with an imperfect process that 
doesn't cover all the criteria and working toward a better process, that 
may or may not cover all of the criteria.

So I have created an interim ~top 30 list:

Most of these bugs are deserving of attention for multiple reasons. I 
used criteria #1 and #2. For #3 I have merely eyeballed and made a best 
judgement that the bug (or it's blockers in the case of a meta) probably 
doesn't require more than a weeks time or major architecutral changes. 
I have not attempted to assess "reproducibility" in this cut.  I created 
this list a week ago when I was still sick, so I don't claim it is 
unbiased or proper in any particular way - it is my interpretation of a 
top 30.  But I will welcome private email as to the suitability of any 
bug to be on the list - and summarize later for the group so it can be 
used to improve whatever future process we put in place.

For my own sanity I excluded large numbers of bugs, like address book 
and filters because the former will hopefully be revamped in the next 
year/in our lifetime, and the later because it has already been heavily 
triaged in recent months due to the efforts of aceman and others. I 
excluded international, build, security either because I have no 
experience in those areas or because the area is highly specialized and 
likely not of interest to the average developer. I excluded 
severity=minor and severity=normal.

Many are dataloss or datalossy. Some are assigned and/or well in 
progress. Many are not.

- I heavily favored bugs that have recent activity (on a sliding scale), 
let's say the last 120 days.
- I have not yet examined most hang and performance bugs - but I am 
pretty sure we have no hang bugs that are pervasive amongst users
- I did not use prior bug lists (for example paper-cuts or various 
wanted flags) to either include or exclude any bugs.

Use in good health.


More information about the tb-planning mailing list