improved test reliability

Robert O'Callahan robert at ocallahan.org
Sun Jul 6 21:01:52 PDT 2014


On Mon, Jul 7, 2014 at 3:59 PM, Chris Jones <jones.chris.g at gmail.com> wrote:

> I finished three runs and saw intermittent failures on 2 tests (two
> separate runs).  One of them I've seen before, one I haven't.  They seem
> related though.  Have to run off, will file and try to debug when I have
> time.
>
> Out of curiosity, which kernel/microarch are you testing on?  I'm
> surprised we're seeing such different results.  (I'm on
> 3.14.2-200.fc20.x86_64 / Ivy Bridge.)
>

Ivy Bridge, host kernel 3.12.7-300.fc20.x86_64.

Also, the ctest harness uses historical test run times to order the tests
> for minimum suite run-time.  That's what you want for everyday dev, but for
> catching extreme edge case failures it's not so much.  Blowing away your
> objdir in between test runs will also destroy the run history and may
> inject some more nondeterminism in the test run sequence.
>

That's interesting!

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofooooolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rr-dev/attachments/20140707/655835f2/attachment.html>


More information about the rr-dev mailing list