<div dir="ltr"><div><div><div><div>Hi all,<br><br></div>we built this dashboard with the intent
of getting a summary of several crash rates and types, with a special
focus on content crash rates since the introduction of Electrolysis.<br><br></div>The increase in crash rates is expected, due to the correlation of content crashes with e10s adoption. As seen here:<a href="https://metrics.mozilla.com/protected/sguha/crashgraphs/#crash-rates-e10s" target="_blank"> https://metrics.mozilla.com/<wbr>protected/sguha/crashgraphs/#<wbr>crash-rates-e10s</a>,
content crashes are the main difference between e10s and non-e10s
users, while main and plugin crashes are not dissimilar between the two
groups (click on the buttons on the right-hand-side to toggle the crash types to compare between the two groups). Therefore, I would suggest waiting until e10s adoption stabilizes in order to see whether crash rates even out as well.<br><div><br>In addition, content crashes are more frequent
than the other two types of crashes (maybe because they are dependent on
how many tabs users have open?). This causes the time between crashes to
decrease: since more users have e10s enabled, more crashes are being
recorded, so the average time in-between crashes decreases.<br><br></div><div>The
data collection that was used to build this dashboard allows us to see
that this is indeed happening: more content crashes are happening (and therefore general crash rates are increasing). This gives us a great opportunity to look
into how to handle this, and try to understand what exactly is causing
so many content crashes to happen in the first place.<br><br></div>I'm
happy to answer any more questions. I am working towards making the github repository public so that you can take a look exactly how all these metrics are being calculated (previous commits contain private Mozilla information such as weekly active users, so I cannot make the current repository public).<br><br></div>Best<br></div><div><div><div class="gmail-yj6qo gmail-ajU"><div id="gmail-:xk" class="gmail-ajR" tabindex="0"><img class="gmail-ajT" src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif"></div></div></div></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Andre Duarte<br></div><div>irc; aduarte<br></div><br></div></div></div></div></div>
<br><div class="gmail_quote">On Tue, Feb 28, 2017 at 12:03 PM, Saptarshi Guha <span dir="ltr"><<a href="mailto:sguha@mozilla.com" target="_blank">sguha@mozilla.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace">Firstly, wrong link and now correct:</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default"><font face="monospace, monospace"><a href="https://metrics.mozilla.com/protected/sguha/crashgraphs/" target="_blank">https://metrics.mozilla.com/<wbr>protected/sguha/crashgraphs/</a></font><br></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">(behind LDAP)</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">Most of the increase is coming from content-crashes. Also E10s</font></div><div class="gmail_default"><font face="monospace, monospace">has moved a lot of crashes into content-crashes. </font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div><div class="gmail_default"><font face="monospace, monospace">Andre, would you like to elaborate?</font></div><div class="gmail_default"><font face="monospace, monospace">Cheers</font></div><div class="gmail_default"><font face="monospace, monospace"><br></font></div></div><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Tue, Feb 28, 2017 at 11:46 AM, John Jensen <span dir="ltr"><<a href="mailto:jjensen@mozilla.com" target="_blank">jjensen@mozilla.com</a>></span> wrote:<br></span><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small">Congratulations for producing an information-filled, easy-to-digest dashboard, but ....<br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small"><br>Wow. Crash rates have increased 10x since September!?<br><br>> Content crashes have been steadily increasing since September 2016.<br>> Content crashes for profiles that have e10s enabled are significantly higher (almost 20x) than for those that don't have it enabled,<br>> around 8-12% of weekly active Firefox users have experienced any type of crash.<br>> The number of new profiles (created that week) that crashed is increasing. <br>> The number of hours between crashes for users who have had a previous crash in their history is decreasing. This means that crashes (of any type) are becoming more frequent in general.<br>> Over 50% of Firefox users who crash per week have had their previous crash within 7 days.<br><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small">There is only alarming news on this dashboard. That's fine, but please correct me if I am wrong.<br><br>I know it's not its objective, but are there better data sources/analytic tools now to understand key drivers of these crashes? Is there a dashboard, say, showing a "burndown" chart of fixes?<br><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small">John<br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small"><br><br><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_3945699148583787522h5">On 28 February 2017 at 11:23, Saptarshi Guha <span dir="ltr"><<a href="mailto:sguha@mozilla.com" target="_blank">sguha@mozilla.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_3945699148583787522h5"><div dir="ltr"><div class="gmail_default"><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">Happy to present the crashgraphs dashboard. A high level dashboard that captures</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">- average profile crash rate (average across profiles crash rates)</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">- % of profiles crashing in the last week</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">- time between crashes for profiles that have another crash in their history</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"> (history is lifetime!)</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">- hours and days between crashes (for profiles with 2+ crashes). Lower is *bad*</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">We capture two things i've been looking for</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">- a profile level view of crash (% of profiles experiencing a crash)</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">- a software engineering level view of crash (hours used across crashes)</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">This is high level so not as detailed as arwwestableyet.</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><span style="font-size:12.8px">Many thanks to Andre Duarte and Connor Ameres for working and designing this.</span><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">Your comments welcome!</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default"><font face="monospace, monospace"><span style="font-size:12.8px"><a href="https://people-mozilla.org/~sguha/crashgraphs/" target="_blank">https://people-mozilla.org/~sg<wbr>uha/crashgraphs/</a></span></font><br></div><div class="gmail_default"><font face="monospace, monospace"><span style="font-size:12.8px"><br></span></font></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">Regard</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">saptarsi</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px">Appendix</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:12.8px"><div class="gmail_default" style="font-size:12.8px"><br></div><div class="gmail_default" style="font-size:12.8px">Also since we use main_summary, we cannot eliminate shutdown crashes. That would be a nice metric to include in main_summary itself.</div><div class="gmail_default" style="font-size:12.8px"><br></div><div class="gmail_default" style="font-size:12.8px">That said, on average shutdown crashes is a fairly constant fraction of total content crashes [1] (the latter measured in main_summary). That is on average. For a given profile with 5 content crashes(as per main_summary), it's tough to say how many are shutdown crashes</div><div class="gmail_default" style="font-size:12.8px"><br></div><div class="gmail_default" style="font-size:12.8px">[1] <a href="https://docs.google.com/document/d/1jzcEPI4NLlar102kS1WBv1cgnxDULYQNl4VawmQEr5c/edit#heading=h.imxtnojofajm" target="_blank">https://docs.google.com/do<wbr>cument/d/1jzcEPI4NLlar102kS1WB<wbr>v1cgnxDULYQNl4VawmQEr5c/edit#h<wbr>eading=h.imxtnojofajm</a></div></div></div></div>
<br></div></div>______________________________<wbr>_________________<br>
fhr-dev mailing list<br>
<a href="mailto:fhr-dev@mozilla.org" target="_blank">fhr-dev@mozilla.org</a><br>
<a href="https://mail.mozilla.org/listinfo/fhr-dev" rel="noreferrer" target="_blank">https://mail.mozilla.org/listi<wbr>nfo/fhr-dev</a><br>
<br></blockquote></div><span class="m_3945699148583787522HOEnZb"><font color="#888888"><br><br clear="all"><br>-- <br><div class="m_3945699148583787522m_-6514734006321143749gmail_signature" data-smartmail="gmail_signature">John Jensen<br><a href="mailto:jjensen@mozilla.com" target="_blank">jjensen@mozilla.com</a><br>Director, Organization Strategy</div>
</font></span></div>
</blockquote></div></div></div><br></div>
</blockquote></div><br></div>