FHR & Telemetry unification: client-side and spec status

[Georg Fritzsche]

As previously discussed, we want to unify the FHR & Telemetry data to use a common system. This is going from the initial plan here:
https://docs.google.com/a/mozilla.com/document/d/1IGpzsYGi_sq3YFQDAPyKOkU_BKvXAC95fZYA2i4ceVs/

From the client-side, we are moving FHR data over to Telemetry and extend & change Telemetry so it covers our needs. This work is targeting Firefox 38.

The main bug for the client-side dependency tree is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1122515

We are doing this in multiple phases:
* Phase 1 (bug 1040800):
  * was reduced to adding a stable client id to telemetry
* Phase 2 (bug 1069869):
  * specifications & format design and
  * the things we need to change in the common Telemetry implementation
* Phase 3 (bug 1120356):
  * remaining data migrations to have data equivalency to FHR
  * other open work that should go into Fx 38 and doesn't directly block the data migrations
* Phase 4 (bug 1122482):
  * contains work that doesn't block for Fx 38

Phase 1 is already complete.
We are now basically done with the design & specification parts of phase 2:
* Bug 1120981 - Properly specify a common Ping format
* Bug 1069880 - Redesign telemetry environment data
* Bug 1120982 - Redesign the main ping format (session ping)

We will have different ping types, e.g. "main" to cover the current telemetry data etc., "activation" to send a base set of data after installation, "upgrade", etc.
All pings will have a basic set of info (bug 1120981) and can additionally submit a stable client id and the environment (bug 1069880).
The actual data for the specific ping type will be then submitted in the payload field. For the "main" ping, we have a basic specification in bug 1120982.

So, as a simple example ping:
{
    version: 2,
    type: "main", // this has the Telemetry payload data
    ..., // some fields common to all pings
    clientId: "5768d94a-ca83-459d-9457-17ab6921b24f", // opt-in per ping type
    environment: { ... }, // opt-in, information about the session environment like app-info & HW
    payload: { ... }, // the actual ping-specific data like histograms for the "main" ping.
                            // This is mostly the current telemetry payload.
}

We identified a set of bugs that we could use help on here:
https://etherpad.mozilla.org/52byZfRPJk
These bugs don't really require changes in the core implementations. Especially the data migration bugs in phase 3 should be relatively straight-forward to do after the phase 2 bugs, re-using etc. existing implementations or adding slightly different new ones.

Georg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fhr-dev/attachments/20150116/13275230/attachment.html>