<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Replies inline<br>
<pre class="moz-signature" cols="72">Tyler Downer
User Advocate for All Things Firefoxy</pre>
On 8/16/13 9:13 AM, William Pietri wrote:<br>
</div>
<blockquote cite="mid:520E4F8D.10403@scissor.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div class="moz-cite-prefix">Hi, Gavin. Sorry if this is coming
across as hostile; I'm not trying to pick on you.<br>
<br>
It's my long-held belief that if there are problems in a
software organization, they're systemic; individuals are doing
the best they can.<br>
<br>
On 08/16/2013 08:20 AM, Gavin Sharp wrote:<br>
</div>
<blockquote
cite="mid:CAHBT5m0J6Za9-3P21HHquEEFYgaREYgxiKSodeM7xQZay8wgLQ@mail.gmail.com"
type="cite">
<pre wrap="">On Fri, Aug 16, 2013 at 3:15 AM, William Pietri <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:william@scissor.com"><william@scissor.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Ok. So it is fair to say that no data was used in estimating the impact of
removing this feature?
</pre>
</blockquote>
<pre wrap="">No, I don't think that's a fair characterization, though it depends
somewhat on what kind of "data" you mean. Experience shipping
software, insight from user research (see e.g.
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://blog.mozilla.org/ux/2012/12/distributed-qualitative-analysis-for-the-firefox-behavioral-segmentation-study/">https://blog.mozilla.org/ux/2012/12/distributed-qualitative-analysis-for-the-firefox-behavioral-segmentation-study/</a>),
and expertise combined with anecdotal data can often be surprisingly
sufficient for making the right decisions. But these things are very
frequently discounted entirely, because they always involve some
degree of subjectivity. They can also be subject to bias, but probably
not more so than "hard" data (or our interpretation of it).</pre>
</blockquote>
<br>
I agree that when intuition is what you've got, that's what you
should go with. But in this case, by "data" I mean what is
traditionally meant by the word: reliable numbers.<br>
<br>
So in this case, it sounds like there was no data available to
you, so you made an expert judgment call. Is that fair to say?<br>
</blockquote>
No it isn't. This change was the first part of a series of fixes by
Project Squeaky, <a class="moz-txt-link-freetext" href="https://wiki.mozilla.org/AMO/Squeaky">https://wiki.mozilla.org/AMO/Squeaky</a>, in an
initiative to harden Firefox against malicious and Adware Add-ons,
which are a very severe problem for millions of Firefox users. We
didn't just pull this change out of anecdotal data alone, but it is
the result of a lot of research by the User Advocacy and AMO Teams.
I can't share the data in a public list, but we do know that
Malicious add-ons are a very real problem for users, and often the
most visible impact of add-on hijacking is that the keyword.url has
been changed from default, for two reasons:<br>
1. Every malicious add-on we have seen changes this pref.<br>
2. There is no visible UI to change this pref, meaning that very few
users would actually have changed it on their own. Plus, users who
have had their keyword.url hijacked have no visible way to reset it.<br>
<br>
This the decision to consolidate search prefs, to both remove an
attack vector and to make it easier for hijacked users to Reset
their prefs to whatever they want.<br>
<blockquote cite="mid:520E4F8D.10403@scissor.com" type="cite"> <br>
<br>
<blockquote
cite="mid:CAHBT5m0J6Za9-3P21HHquEEFYgaREYgxiKSodeM7xQZay8wgLQ@mail.gmail.com"
type="cite">
<blockquote type="cite">
<pre wrap="">Also, as to the approximate number of users affected, it sounds like you
estimated it in the range of "a lot of people", but that could be anywhere
from thousands to millions? Could it be tens of millions?
</pre>
</blockquote>
<pre wrap="">It seems unlikely that you'll trust my judgement here, so I won't try
to convince you, but I think it quite unlikely that this had a
negative impact on millions of users. We do not know for sure, of
course.</pre>
</blockquote>
<br>
Well, it's nothing personal against your judgment; I'm sure it's
quite good, especially working from no real data on this. I'm just
trying to put error bars on the number of affected users.<br>
<br>
One way to work at this is to work backwards from impact. So if
you work this equation:<br>
<br>
<blockquote># of users who use this feature *<br>
% of users who have upgraded to FF23 *<br>
% of upgraders who consciously recognize the breakage *<br>
% of recognizers who bother to search *<br>
% of searchers who find what happened *<br>
% of those people who find the add-on *<br>
% of people who bother to install the add-on =<br>
538<br>
</blockquote>
You can try your own percentages, and they'll probably be more
correct than mine, but when I play with this I get pretty large
numbers.<br>
<br>
So from that, it seems fair to say that the number of affected
users is thousands to millions, but tens of millions is out of
bounds, yes?<br>
<br>
</blockquote>
It depends on how you define "affected". If you mean "Users who had
their keyword.url hijacked and now have had it reset" then yes, lots
of users have been affected, mostly positively. If you mean "Users
who want multiple search engines and now have to install an add-on
to restore that functionality", then you are talking thousands of
users at the most.<br>
<blockquote cite="mid:520E4F8D.10403@scissor.com" type="cite">
<blockquote
cite="mid:CAHBT5m0J6Za9-3P21HHquEEFYgaREYgxiKSodeM7xQZay8wgLQ@mail.gmail.com"
type="cite">
<blockquote type="cite">
<pre wrap="">But I'm puzzled by what you write. I thought the theory of this feature was
that people with hijacked url bar search needed to control that so much that
it was worth killing a feature. But if most people can't discover the search
bar dropdown, how is this solving the problem?
</pre>
</blockquote>
<pre wrap="">The feature was beneficial in two ways:
- it consolidated search settings and removed the use of a preference
(keyword.URL) that we have evidence was being widely abused. This is
the primary source of hijacking-protection.
- because the search settings were consolidated and are all controlled
by user-visible UI (the search bar), in any remaining cases where a
third party changes your search settings non-maliciously, users now
have the ability to easily revert it without having to resort to tools
like <a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://addons.mozilla.org/en-US/firefox/addon/searchreset/">https://addons.mozilla.org/en-US/firefox/addon/searchreset/</a>
Lack of discoverability of the search bar dropdown only negatively
impacts that second benefit, and while it might mitigate it somewhat,
overall the impact was still positive (some users do know how to use
the dropdown).
</pre>
</blockquote>
<br>
Could you help me understand how the first benefit is different
than the second? I'm not seeing any lasting user benefit to point
1 on its own; hijackers will presumably quickly retarget.<br>
</blockquote>
The search dropdown is at least more discoverable than <a class="moz-txt-link-freetext" href="about:config">about:config</a>,
so we made the story better. Also, don't think this is the end of
project squeaky's efforts to harden Firefox, we've got more plans in
the works and you can find them all from the wiki.<br>
<blockquote cite="mid:520E4F8D.10403@scissor.com" type="cite"> <br>
William<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
firefox-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:firefox-dev@mozilla.org">firefox-dev@mozilla.org</a>
<a class="moz-txt-link-freetext" href="https://mail.mozilla.org/listinfo/firefox-dev">https://mail.mozilla.org/listinfo/firefox-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>