Suggestions for the new unified FHR/Telemetry/Experiment ping

[Vladan Djeric]

I'm going to answer questions raised by others first in the next couple of
messages, and then I'm going to write a much longer message covering all
the issues that arise if we choose the original approach (reset Telemetry
on new ping) or my suggested approach (allow Telemetry to accumulate
without resetting).


>    - Whenever a new ping is started, all the FHR & Telemetry measurements
>    for the current session will be reset
>
> My assumptions was that we will not reset. There is Telemetry data that is
> not meaningful when reset, so i could only imagine opt-in for specific
> probes to be reset (e.g. add a “daily” property to histograms if we need
> that).
> Do we have a good argument for resetting all of them in the first place?
> Most of your concerns result from that.
>

As others have explained, the FHR/Telemetry v4 plan specifies resetting
both Telemetry & FHR data every time a new ping is created.


>
>    - In the new system, Firefox starts collecting a new ping whenever:
>       1. a new Firefox session is started
>       2. a new day has begun (not sure if it's every 24 hours of uptime,
>       or if it's based on local time e.g. midnight local time)
>
> That was a partially open question, midnight local time is what we will do
> unless there is useful counter arguments.
>

If FHR analyses are based around local time, we have to start pings on
midnight local time. This raises some challenges for how we store the
submitted pings server-side. The S3 archives are currently organized by
submission date not (local) *collection* date. This discussion is probably
better to have in person with Mark.


>
>    1. Record mid-session environment changes (add-ons and
>    TelemetryExperiments) in a special section in the ping.
>       - For each such environment change, document the change in the
>       section and also attach a snapshot of the Telemetry & FHR data at the time
>       of the change
>       - After the snapshot is saved, reset Telemetry and FHR measurements
>       for the current session. In other words, snapshot & then build up a diff
>       - For each additional environment change during the same session,
>       just repeat and append to the new section
>       - Telemetry backend scripts (dashboard, regression detector etc)
>       can just ignore experiment/add-on change pings
>
> What would the snapshots diff against? Diffing going back from the final
> ping? Also, any diffing client-side adds complexity and potential issue -
> i’d prefer to avoid that if possible.
>

What I meant is, when an environment changes mid-session, take a snapshot
of the current Telemetry histograms and back it up in a special section of
the ping. That's the snapshot. *Then reset the Telemetry histograms, and
simply continue the session.*
The measurements stored in the newly *cleared* Telemetry histograms
represent a "diff" against that snapshot taken when the environment
changed. In other words, the histograms now store measurements taken since
the environment changed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fhr-dev/attachments/20150127/7cfd5094/attachment.html>