Breaking url-bar search

Gavin Sharp gavin at gavinsharp.com
Fri Aug 16 15:20:49 UTC 2013


On Fri, Aug 16, 2013 at 3:15 AM, William Pietri <william at scissor.com> wrote:
> Ok. So it is fair to say that no data was used in estimating the impact of
> removing this feature?

No, I don't think that's a fair characterization, though it depends
somewhat on what kind of "data" you mean. Experience shipping
software, insight from user research (see e.g.
https://blog.mozilla.org/ux/2012/12/distributed-qualitative-analysis-for-the-firefox-behavioral-segmentation-study/),
and expertise combined with anecdotal data can often be surprisingly
sufficient for making the right decisions. But these things are very
frequently discounted entirely, because they always involve some
degree of subjectivity. They can also be subject to bias, but probably
not more so than "hard" data (or our interpretation of it).

(This isn't to say that we shouldn't be doing a better job of
collecting data where we can. Historically that has been difficult to
do for us - we care a lot about user privacy, which makes collecting
data more challenging, and we just haven't had the infrastructure
required to collect and also properly analyze all the data that we'd
like. But we've been investing in improvements, people and tech, and
are in a much better place now than we've ever been. With tools like
FHR, which we're just starting to benefit from, we will be able to be
better informed in the future.)

> Also, as to the approximate number of users affected, it sounds like you
> estimated it in the range of "a lot of people", but that could be anywhere
> from thousands to millions? Could it be tens of millions?

It seems unlikely that you'll trust my judgement here, so I won't try
to convince you, but I think it quite unlikely that this had a
negative impact on millions of users. We do not know for sure, of
course.

> But I'm puzzled by what you write. I thought the theory of this feature was
> that people with hijacked url bar search needed to control that so much that
> it was worth killing a feature. But if most people can't discover the search
> bar dropdown, how is this solving the problem?

The feature was beneficial in two ways:
- it consolidated search settings and removed the use of a preference
(keyword.URL) that we have evidence was being widely abused. This is
the primary source of hijacking-protection.
- because the search settings were consolidated and are all controlled
by user-visible UI (the search bar), in any remaining cases where a
third party changes your search settings non-maliciously, users now
have the ability to easily revert it without having to resort to tools
like https://addons.mozilla.org/en-US/firefox/addon/searchreset/

Lack of discoverability of the search bar dropdown only negatively
impacts that second benefit, and while it might mitigate it somewhat,
overall the impact was still positive (some users do know how to use
the dropdown).

Gavin



More information about the firefox-dev mailing list