<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Looks relevant to FxA perf testing.<br><div><br><div>Begin forwarded message:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; color:rgba(0, 0, 0, 1.0);"><b>From: </b></span><span style="font-family:'Helvetica';">Mark Mayo <<a href="mailto:mmayo@mozilla.com">mmayo@mozilla.com</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; color:rgba(0, 0, 0, 1.0);"><b>Subject: </b></span><span style="font-family:'Helvetica';"><b>Fwd: Marketplace Performance Tests</b><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; color:rgba(0, 0, 0, 1.0);"><b>Date: </b></span><span style="font-family:'Helvetica';">June 24, 2014 at 10:22:29 PM PDT<br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; color:rgba(0, 0, 0, 1.0);"><b>To: </b></span><span style="font-family:'Helvetica';">"<a href="mailto:cloudservices-managers@mozilla.com">cloudservices-managers@mozilla.com</a>" <<a href="mailto:cloudservices-managers@mozilla.com">cloudservices-managers@mozilla.com</a>><br></span></div><br><div><div dir="ltr">note sure how many of our folks are on the webdev mailing list, but this one looked 'relevant to our interests'.<br><br>-m<br><div><br><div class="gmail_quote">---------- Forwarded message ----------<br>
From: <b class="gmail_sendername">Christopher Van</b> <span dir="ltr"><<a href="mailto:cvan@mozilla.com">cvan@mozilla.com</a>></span><br>Date: Tue, Jun 24, 2014 at 10:14 PM<br>Subject: Marketplace Performance Tests<br>
To: <a href="mailto:marketplace-team@mozilla.org">marketplace-team@mozilla.org</a>, WebDev <<a href="mailto:webdev@mozilla.com">webdev@mozilla.com</a>><br><br><br>I'm trying out this service called SpeedCurve. It uses WebPagetest behind the scenes but with an actually decent UI.<br>
<br>
It runs 108 "synthetic" tests across Amazon servers to do perf tests across all the major browsers. (A good read: <a href="http://speedcurve.uservoice.com/knowledgebase/articles/355134-synthetic-vs-real-user-monitoring-rum" target="_blank">http://speedcurve.uservoice.com/knowledgebase/articles/355134-synthetic-vs-real-user-monitoring-rum</a>)<br>
<br>
Here are stats comparing Marketplace prod vs. Google Play vs. Google Chrome Web Store:<br>
<br>
<a href="http://speedcurve.com/share/bae1o9hxq2nvetwzno8o6996zr4mc2/" target="_blank">http://speedcurve.com/share/bae1o9hxq2nvetwzno8o6996zr4mc2/</a><br>
<br>
Yeah, that's right - we are pretty darn fast even by Google standards.<br>
<br>
And Marketplace prod vs. stage vs. dev:<br>
<br>
<a href="http://speedcurve.com/share/dfbj4g5ltp43k30xt17fj5zvm6lllj/" target="_blank">http://speedcurve.com/share/dfbj4g5ltp43k30xt17fj5zvm6lllj/</a><br>
<br>
Keep an eye on these pages as they get updated daily and watch how fast we get.<br>
<br>
I had written my own perf measurement tool (called phantomHAR), but this thing seems to work and looks great (although it's unfortunately not open source yet).<br>
<br>
phantomHAR fetches a URL (based on a manual POST or GitHub webhook), generates a HAR (the JSON that can render waterfall graphs you see in the Network tab of your favourite browser's Developer Tools), tallies up all the response sizes and times, and then the front-end renders graphs of the response sizes, response times, and number of requests for each type of request (e.g., HTML document/JS/CSS/inline image/CSS image/etc.). I use a fantastic tool called chromeHAR <<a href="https://github.com/ericduran/chromeHAR" target="_blank">https://github.com/ericduran/chromeHAR</a>> to render the HARs (meaning, when you click on a particular date, you can see the waterfall graph of all the network requests made with the request/response headers, response sizes, tabs for each request type, etc. - just like in your browser's devtools).<br>
<br>
This is what an early version of what phantomHAR looks like:<br>
<br>
<a href="http://people.mozilla.org/~cwiemeersch/phantomhar/" target="_blank">http://people.mozilla.org/~cwiemeersch/phantomhar/</a><br>
<br>
I'm going to continue work on phantomHAR because it gives me the information I want — perf metrics after I push code to our staging/dev servers. I like that because when I remove 300 lines of JS, remove a webfont, or upgrade jQuery, I want to immediately see how much faster/slower I affected the site and how many bytes I saved/wasted with that change. Keeping an audit trail of the network requests is imperative if you're going to ever know if you're getting faster or slower. Things like NewRelic help this on the server, but there really aren't any good tools that help developers diff the network requests each time new code is pushed to their servers.<br>
<br>
There's a similar tool that also uses PhantomJS to do perf profiling. It's called phantomas <<a href="https://github.com/macbre/phantomas" target="_blank">https://github.com/macbre/phantomas</a>> and it does a heck of a lot, and I've been contributing to it. There's a grunt plugin called grunt-phantomas <<a href="https://github.com/stefanjudis/grunt-phantomas" target="_blank">https://github.com/stefanjudis/grunt-phantomas</a>> that can generate many, many pretty graphs from the resulting JSON (see <a href="http://cvan.io/marketplace-perfboard/prod/" target="_blank">http://cvan.io/marketplace-perfboard/prod/</a> for examples). Besides showing too much information, its HAR generator <<a href="https://github.com/macbre/phantomas/blob/master/modules/har/har.js" target="_blank">https://github.com/macbre/phantomas/blob/master/modules/har/har.js</a>> generates incorrect filesizes because of an age-old PhantomJS bug <<a href="https://github.com/ariya/phantomjs/issues/10156" target="_blank">https://github.com/ariya/phantomjs/issues/10156</a>> <<a href="https://github.com/ariya/phantomjs/issues/10158" target="_blank">https://github.com/ariya/phantomjs/issues/10158</a>>. YSlow, netsniff.js, and many other PhantomJS-based scripts are also inaccurate. I've worked around these issues with phantomHAR (fastHAR-api) until the fix <<a href="https://github.com/ariya/phantomjs/pull/11484" target="_blank">https://github.com/ariya/phantomjs/pull/11484</a>> gets upstreamed to phantomJS.<br>
<br>
Anyway, if people are interested in contributing to phantomHAR, holler at me because I could use some help with it (it's all NodeJS/JS). I'll be consolidating the repos but for now there are three of them:<br>
<br>
<a href="https://github.com/cvan/phantomHAR" target="_blank">https://github.com/cvan/phantomHAR</a><br>
<a href="https://github.com/cvan/fastHAR-api" target="_blank">https://github.com/cvan/fastHAR-api</a><br>
<a href="https://github.com/cvan/fastHAR" target="_blank">https://github.com/cvan/fastHAR</a><br>
<br>
When officially launched, phantomHAR live at <a href="http://arewefast.com/arewefast.org" target="_blank">arewefast.com/arewefast.org</a>.<br>
<br>
Anyway, keep an eye on the SpeedCurve URLs. And reach out to me if you're interesting in helping me with front-end perf monitoring.<br>
</div><br></div></div>
</div></blockquote></div><br></body></html>