<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<div>
<p>Hi everybody,</p>
<p>During December 2019 we released the first version of <a
href="https://firefox-source-docs.mozilla.org/tools/lint/linters/perfdocs.html"
target="_blank">PerfDocs</a> which initially verified if all
the Raptor performance tests have a test description. Last week
we (me [:<span><span class="gmail-mx_SenderProfile_name
gmail-mx_Username_color2">alexandrui</span></span>] and
Gregory Mierzwinski [:sparky]) made another release that added
the document generation feature.</p>
<p><a
href="https://firefox-source-docs.mozilla.org/tools/lint/linters/perfdocs.html"
target="_blank">PerfDocs</a> was born for the need of having a
centralized way for the test metadata (e.g. descriptions). It
can be used with the command <font face="Courier New, Courier,
monospace">./mach lint -l perfdocs</font> and it has 2 steps:
verification and generation (<font face="Courier New, Courier,
monospace">--fix</font> ing). While the verification is
performed by default, the generation requires passing the <font
face="Courier New, Courier, monospace">--fix</font> optional
argument. What it does is to search in the tree for any <font
face="Courier New, Courier, monospace">perfdocs</font>
directory that has certain content (2 files). The first content
file is <font face="Courier New, Courier, monospace">config.yml</font>
which holds the location of the framework's test manifests and
the descriptions for the tests that are found by PerfDocs using
the manifests. The second file that is provided in the folders,
called<font face="Courier New, Courier, monospace"> index.rst</font>,
determines where/how the test information will be displayed in
the <a href="https://firefox-source-docs.mozilla.org/"
target="_blank">mozilla source tree docs</a> (under
Testing/Performance Testing) and lets users add extra framework
information if needed. See <a
href="https://dxr.mozilla.org/mozilla-central/source/testing/raptor/raptor/perfdocs">here</a>
for an example folder with Raptor. You can find the
documentation generated from that specification <a
moz-do-not-send="true"
href="https://firefox-source-docs.mozilla.org/testing/perfdocs/index.html">here</a>.<br>
</p>
<p>One thing to note is the <font face="Courier New, Courier,
monospace">./mach lint -l perfdocs</font> linter will FAIL if
any test/suite present in the framework manifest doesn't have an
associated description (individual or pattern-matched). If there
are no tests missing a description and all descriptions in
perfdocs/config.yml are matching at least one test then the
linting will pass. It will also let the user know if the
documentation needs to be regenerated. When <font face="Courier
New, Courier, monospace">--fix</font> is provided as an
argument, then verification will be performed along with
document (re-)generation. In the case of a linting failure, you
will need to update <font face="Courier New, Courier,
monospace">config.yml</font>.<br>
</p>
<p>Example:<br>
</p>
<p>Given tests: <font face="Courier New, Courier, monospace">raptor-tp6-amazon-firefox</font>,
<font face="Courier New, Courier, monospace">raptor-tp6-amazon-chrome</font>,
<font face="Courier New, Courier, monospace">raptor-tp6-amazon-chromium</font>
and their <font face="Courier New, Courier, monospace">-cold</font>
variants. </p>
<p>Individual matching:</p>
<p> <font face="Courier New, Courier, monospace">raptor-tp6-amazon-firefox:
"Warm desktop page-load performance test for amazon on firefox
using WebExtension."</font></p>
<p>Pattern matching:</p>
<p> <font face="Courier New, Courier, monospace">raptor-tp6-amazon:
"Cold and warm desktop page-load performance tests for amazon
on firefox, chrome, and chromium using WebExtension."</font></p>
</div>
<div><br>
<p>Many thanks to the perftest team who helped on reviewing this
all along the way.<br>
</p>
<p>For any questions do not hesitate to contact us.</p>
<p><br>
</p>
<p>Alex<br>
</p>
</div>
</body>
</html>