<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Mon, Apr 3, 2017 at 12:19 PM, Dave Townsend <span dir="ltr"><<a href="mailto:dtownsend@mozilla.com" target="_blank">dtownsend@mozilla.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><span class="gmail-">On Mon, Apr 3, 2017 at 9:16 AM, Ehsan Akhgari <span dir="ltr"><<a href="mailto:ehsan.akhgari@gmail.com" target="_blank">ehsan.akhgari@gmail.com</a>></span> wrote:<br></span><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br><div dir="ltr"><span><div class="gmail_extra"><div class="gmail_quote">On Mon, Apr 3, 2017 at 12:11 PM, Mark Banner <span dir="ltr"><<a href="mailto:mbanner@mozilla.com" target="_blank">mbanner@mozilla.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span>
On 03/04/2017 16:45, Ehsan Akhgari wrote:<br>
<blockquote type="cite">
<div dir="ltr">
<ul>
<li>Are there some profile links showing what the
startup/shutdown of the browser looks like with this
extension installed?</li>
</ul>
</div>
</blockquote></span>
I'll let Ian answer most of this. What I can say that has definitely
been done are Talos runs which showed no regressions.<br>
<br>
<a class="gmail-m_757071412928977815m_-220310353351875942m_2965181148551821944moz-txt-link-freetext" href="https://github.com/mozilla-services/screenshots/issues/2317" target="_blank">https://github.com/mozilla-ser<wbr>vices/screenshots/issues/2317</a><span class="gmail-m_757071412928977815m_-220310353351875942HOEnZb"></span><br></div>
</blockquote></div><br></div></span><div class="gmail_extra">Talos should be thought of more as a smoke detector for this kind of feature. That it has not shown any regressions should be a sign that the building isn't on fire, not necessarily that the feature is ready to be shipped. :-)<br clear="all"></div><div class="gmail_extra"><br></div><div class="gmail_extra">Unfortunately at this time we have no automated tools that can analyze the impact of a few front-end feature without a lot of manual performance investigation.<br></div></div></blockquote><div><br></div></span><div>Do we have good documentation on how to do the manual performance investigation</div></div></div></div></blockquote><div><br></div><div>The basics are pretty simple: go to <a href="https://perf-html.io">https://perf-html.io</a>, install the profiler add-on and start profiling using the instructions on that page! We don't unfortunately have a lot of great documentation at this point due to the lack of bandwidth to produce it. Sorry about that. We're trying our best to move fast on all fronts here but we're really starved for choosing what to work on. :-(<br><br></div><div>Last Friday at the Quantum Flow work week here in Toronto, I gave an informal talk about how to use the Gecko Profiler since people were interested in seeing how I use it. I didn't have any slides or any specific profiling scenario prepared so I decided to profile Firefox live and delve into a couple of scenarios in the parent and content processes. The presentation is being uploaded right now and will be available at this link once finished: <<a href="https://air.mozilla.org/gecko-profiler-introduction/">https://air.mozilla.org/gecko-profiler-introduction/</a>>. I hope this will be a good starting point. If people have questions about how to use the profiler and how to read the profiles, I and others hanging out in #flow would be happy to help answer questions!<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> and a plan for adding the automated testing we need to avoid it? <br></div></div></div></div>
</blockquote></div><br></div><div class="gmail_extra">Automated testing for this is easier said than done... :/ In general building a tool that can analyze an arbitrary feature without knowing anything about it in advance is a research problem. The reason tools like Talos aren't a suitable measurement utility here is that they have no tests that would directly measure anything that this feature would directly impact, and most of their tests are probably too noisy for any potential impact that the feature _could_ be introducing to show up in any significant way in the test output numbers. This excludes tests like Ts, in theory, but in practice even those tests have been less helpful in preventing startup regressions than I would really like... Why that is the case, I'm not really sure. It could be due to issues in Talos, or in how we use it, or both. Again, using Ts as an example, we have gotten more than twice as bad over the past year on this Ts Windows 32-bit e10s PGO measurement <<a href="https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=%5Bmozilla-central,e394aab72917d169024558cbab33eb4e7e9504e1,1,1%5D">https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=%5Bmozilla-central,e394aab72917d169024558cbab33eb4e7e9504e1,1,1%5D</a>>, so perhaps there is just a death by a thousand cuts effect at play here.<br><br></div><div class="gmail_extra">But for the time being, I think we shouldn't bet the farm on just running tests on Talos, especially for features that don't even get the regular Talos testing that normal features developed on m-c get.<br clear="all"></div><div class="gmail_extra"><br>-- <br><div class="gmail_signature"><div dir="ltr">Ehsan<br></div></div>
</div></div>