Changing the Firefox code that provides "usage hours" numbers

Saptarshi Guha sguha at mozilla.com
Wed Nov 18 22:18:11 UTC 2015


Completely fine with _me_. I stopped using activeticks (from FHR v2), and i
think nobody else does either.
Regards
Saptarshi


On Wed, Nov 18, 2015 at 2:12 PM, Vladan Djeric <vdjeric at mozilla.com> wrote:

> So do you mind if we get rid of activeTicks completely and replace it with
> something more accurate? That would make our work a lot easier
>
> On Wed, Nov 18, 2015 at 5:08 PM, Saptarshi Guha <sguha at mozilla.com> wrote:
>
>> I think we would have always liked to use (and would like to use)
>> activeticks (less skew and variance than totalseconds), but we were informed
>> that the measurement was unreliable/uninformative. So the dashboards
>> reflect totalseconds/hrs.
>>
>> Regards
>> Saptarshi
>>
>>
>> On Wed, Nov 18, 2015 at 2:00 PM, Vladan Djeric <vdjeric at mozilla.com>
>> wrote:
>>
>>> Have the Metrics team's requirements changed since we last discussed
>>> this in 2014? Do you still rely on the current definition of activeTicks?
>>>
>>> We need a more accurate measure of activeTicks for the E10S performance
>>> analysis [1], so this topic is relevant again.
>>>
>>> 1.
>>> https://docs.google.com/document/d/1TyE0BehzYhii3qfmcrfjXlRJL64CcJk0B4Voup4Q0Pg/
>>>
>>> On Fri, Dec 26, 2014 at 6:14 PM, John Jensen <jjensen at mozilla.com>
>>> wrote:
>>>
>>>> Hi Georg,
>>>>
>>>> Just to summarize:
>>>>
>>>> - Thank you for asking for our comments about this issue.
>>>> - We have used activeTicks specifically, and with other measures, to
>>>> model the impact of code changes on user activity. We plan to continue to
>>>> do so in 2015.
>>>> - We understand that this involves some more work for your team, but it
>>>> is our considered opinion and request would prefer that both approaches to
>>>> measuring activeTicks run concurrently for 2-3 releases.
>>>>
>>>> John
>>>>
>>>> On Fri, Dec 26, 2014 at 11:23 AM, Saptarshi Guha <sguha at mozilla.com>
>>>> wrote:
>>>>
>>>>> > I understand being careful about breaking important metrics, but if
>>>>> its not critical right now, it would be great to avoid tacking on slightly
>>>>> awkward work-arounds.
>>>>> I still favor adding a new measurement rather than just re
>>>>> implementing the old.  A lot of our existing numbers are based on this
>>>>> implementation and by using a new variable along slide old one we can
>>>>> hopefully rescale the old values.
>>>>>
>>>>> Is adding a new variable and phasing out the old variable after 2-3
>>>>> versions so undesirable?
>>>>>
>>>>> Regards
>>>>> Saptarshi
>>>>>
>>>>>
>>>>> On Mon, Dec 22, 2014 at 11:20 PM, Georg Fritzsche <
>>>>> gfritzsche at mozilla.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Dec 18, 2014 at 6:40 PM, John Jensen <jjensen at mozilla.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> 1) the activeTicks measure has been used in a number of different
>>>>>>> ways. We've used it as part of an overall "usage index" and to help
>>>>>>> understand which subpopulations have higher or lesser usage than others for
>>>>>>> the Engagement team and others. It was these metrics that, among other
>>>>>>> things, helped uncover the regression that was fixed in bug 959130. If the
>>>>>>> underlying data collection for this metric is going to change, we need to
>>>>>>> be able to understand how so that we can account for it as we report for
>>>>>>> 2014 and potentially for next year.
>>>>>>>
>>>>>>
>>>>>> I only see the usage of firstpaint and sessionrestored times in that
>>>>>> bug, not activeTicks. Am i missing something here?
>>>>>>
>>>>>> 2) this is important because we need to have some overall indicator
>>>>>>> of "usage" beyond session length...
>>>>>>>
>>>>>>
>>>>>> Right now, this indicator is pretty broken on Windows (massively
>>>>>> overreporting), so i'm not sure if this gives much more info then "time
>>>>>> users had their mouse over the window", which doesn't seem too meaningful?
>>>>>>
>>>>>> The other thing is that, from what i know, this is not an important
>>>>>> metric for next year, so my understanding was that we don't have important
>>>>>> consumers of it for that time frame.
>>>>>> I understand being careful about breaking important metrics, but if
>>>>>> its not critical right now, it would be great to avoid tacking on slightly
>>>>>> awkward work-arounds.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> While we are on the subject, we should talk sometime about which
>>>>>>> automated tests are part of Talos or other systems to regularly check the
>>>>>>> correctness of FHR submissions as parts of the codebase change. We need to
>>>>>>> what we can to prevent issues like this from recurring in future.
>>>>>>>
>>>>>>
>>>>>> We do have bug 1111631 for this, but i'm afraid we're short on
>>>>>> resources to get to this soon.
>>>>>>
>>>>>> Georg
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> John Jensen
>>>> Director, Metrics, Mozilla Corporation jjensen at mozilla.com
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fhr-dev/attachments/20151118/de50a93a/attachment-0001.html>


More information about the fhr-dev mailing list