<div dir="ltr"><div><div><div><div><div><div>Think of the per-build data like this:<br><br></div>* our crash rate for FF53 b2 is too goddamn high!<br></div>* We pulled a topcrash list and found a regression bug<br></div>* We fixed it and uplifted it to FF53b4<br></div>* Release drivers want to make sure that FF53b4 has the crash rate reduction that we expected.<br><br></div>In this case (which is very common on all the prerelease channels), showing by date and not by build smooths out the signal we care about most, which is the difference per-build. So what release drivers care about most is: how quickly can we get from releasing beta8 (or RC1) to a reliable crash rate that says we're clear to ship this to release?<br><br></div>--BDS<br><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 1, 2017 at 2:45 PM, Chris Hutten-Czapski <span dir="ltr"><<a href="mailto:chutten@mozilla.com" target="_blank">chutten@mozilla.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>Thank you for correcting my mistake about build_id. Apparently when I was looking for readable versions I took the lack of a bX suffix on beta builds to mean there was no build data, but that was wrong then and wrong now.<br><br></div>So yeah. No problemo, apparently. (Well, except my misapprehension, which should now be rectified.)<br><br></div><div>Anyway, back to wishlisting... The request I can most easily understand is for the existing display to instead use the current and N previous _builds'_ crash counts and kuh to form a "channel health trend line". N may be some fixed, small number per channel (maybe 2, 6, 14, 21), or some function of usage (take all successive previous builds until they contain > Y% of that activity_date's kuh).<br><br></div><div>This sounds fun and useful and is something I feel I understand well enough to file an Issue for and begin working on.<br><br></div><div>The per-build case on the other hand I feel I understand less well. Do the pseudo-timeseries plots as seen on sql.tmo work as well on channels with fewer builds? If instead we display the crash rate for one build as one number (per type of crash), does it need to be plotted at all? Is Harald's view of 52b5 actually a problem? The best display for understanding the health of a new release is... <br></div><div><br></div><div>:chutten<br></div></div></div></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 1, 2017 at 2:08 PM, Benjamin Smedberg <span dir="ltr"><<a href="mailto:benjamin@smedbergs.us" target="_blank">benjamin@smedbergs.us</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Wed, Mar 1, 2017 at 1:40 PM, Chris Hutten-Czapski <span dir="ltr"><<a href="mailto:chutten@mozilla.com" target="_blank">chutten@mozilla.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>So, for each channel, having a line for each of the current and N previous versions' crash rates would be helpful? (where N is small... say 2)<br></div></div></div></blockquote></span><div><br>It's closer. If it's possible to have a single line that is the aggregate, that might smooth out adoption noise.<br> </div><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br><div>crash_aggregates does its aggregation by version, not build, so if crash-rates-per-build is necessary this will require quite a rewrite. (I'll have to join main_summary and crash_summary on dates) If crash-rates-per-version is sufficient (or is worth the effort of exploring), then that can be adopted within the existing architecture.<br></div></div></div></blockquote><div><br></div></span><div>This is incorrect. build_id is one of the dimensions, specifically because we needed it for betas with e10s. Example from stmo:<br><br>activity_date
dimensions
stats
submission_date
24/08/16{"build_id":"201508040<wbr>30204","os_name":"Windows_NT",<wbr>"os_version":"6.2","country":"<wbr>NP","application":"Firefox","a<wbr>rchitecture":"x86-64","build_v<wbr>ersion":"42.0a1","channel":"ni<wbr>ghtly","e10s_enabled":"False"}<wbr>{"usage_hours_squared":0.<wbr>01585779320987654,"main_crashe<wbr>s":0,"content_shutdown_crashes<wbr>":0,"usage_hours":0.1541666666<wbr>6666667,"content_shutdown_cras<wbr>hes_squared":0,"gmplugin_crash<wbr>es":0,"content_crashes":0,"<wbr>content_crashes_squared":0,"<wbr>plugin_crashes":0,"plugin_cras<wbr>hes_squared":0,"ping_count":3,<wbr>"gmplugin_crashes_squared":0,"<wbr>main_crashes_squared":0}25/08/<wbr>16<br><br><br></div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><br></div><div>One thing to consider may be that crash-rates-per-build will look messy on anything pre-Beta. <br></div></div></div></blockquote><div><br></div></span><div>Could be, but here's an example way to present this for nightly: <a href="http://benjamin.smedbergs.us/blog/2013-04-22/graph-of-the-day-empty-minidump-crashes-per-user/" target="_blank">http://benjamin.smedbergs.us/b<wbr>log/2013-04-22/graph-of-the-da<wbr>y-empty-minidump-crashes-per-u<wbr>ser/</a><br><br></div><div>Here's also a draft of something a while back in STMO: <a href="https://sql.telemetry.mozilla.org/queries/192/source#309" target="_blank">https://sql.telemetry.mozilla.<wbr>org/queries/192/source#309</a><br><br></div><div>--BDS</div></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>