Followup: modularity, WebExtensions, and going faster

[Gregory Szorc]

On Thu, Oct 6, 2016 at 5:08 PM, Benjamin Smedberg <benjamin at smedbergs.us>
wrote:

> I spent a week writing a thing about modularity, webextensions, and going
> faster. I think it's important for us to decide the module structure of our
> code especially as we start shipping independent modules/going faster. And
> I believe that having better module structure, boundaries, and
> documentation is critical to our teams being more agile and also attracting
> contributors to the project.
>
> http://benjamin.smedbergs.us/blog/2016-09-03/modularity-and-webextensions/
>
> I personally think that we should double down on WebExtensions as a model
> and start using that for large parts of Firefox. But Andy McKay and Rob
> Helper had some good counter-thoughts and I've asked them to post here to
> elaborate.
>
> In the post I asked everyone to send followups to firefox-dev, so I wanted
> to start a thread here to collect responses. Over the next months I'd like
> this to turn into a firm decision about how we're going to build system
> addons; but I'd like to start by seeing what feedback people have and even
> whether I've framed the problem correctly.
>

Having slept on this post, I have a few more comments.

1) It sounds like you are advocating a massive rewrite of the Firefox
frontend code. It isn't a complete bottoms-up rewrite. But on a scale of 1
to 10, I think it is solidly a 7. Maybe an 8. In terms of effort required,
I think the project is at least on the same scope as e10s. In terms of
people resources and the value associated with people time, this project
will effectively cost tens of millions of dollars and likely take 2+ years.
Where are we going to get the staffing for that? What other features have
to be put on hold while we rewrite core features?

2) The more years of experience I accrue, the more skeptical I am that
significant rewrites or something resembling significant rewrites are
beneficial. The rest of the industry seems to agree that incremental,
targeted refactorings yield better results. I am highly dubious that
refactoring/rewriting significant existing Firefox frontend features to be
WebExtensions will be worthwhile. If we want to implement new features as
WebExtensions or convert existing, self-contained features as
WebExtensions, go for it. But to advocate for proactively changing the
architecture of the existing features sounds scary to me (in fairness, I'm
not sure if you are advocating this or whether you are merely saying we
should strive for a WebExtensions first architecture).

3) If we're talking about a significant change of architecture to the
Firefox frontend and the high costs associated with that change, I'll throw
out an alternative idea: (re)write the frontend in a typed language like
Rust. I argue that a typed language - and Rust especially - would deliver
"well-defined, documented, and enforced module boundaries or interfaces"
better than JavaScript ever will.

4) I have significant concerns about the concept of system add-ons.
Currently, the composition of Firefox (in terms of code) is essentially
deterministic from the commit in mozilla-central it was built from. We know
exactly what Firefox code is running (OK, /should/ be running) on users'
machines. System add-ons fundamentally change that from ~1 combination of
software components to N, where N could be absurdly large (10^3 or larger).
It is hard enough to track down intermittent failures in test automation
with a well-defined software configuration. System add-ons introduce whole
new classes of bugs related to how different versions of add-ons interact
with each other. They require more and more complicated testing
infrastructure to test various combinations of add-ons. This makes assuring
quality harder. System add-ons seem like they add cost to the release
process and undermine stability (even if they may make writing code easier
- but software development is about a whole lifecycle, not just writing the
code).

5) I challenge the necessity that system add-ons or WebExtensions are
necessary to "go faster." There are a host of changes we could make to the
release process to "go faster." System add-ons feel like they are on the
more radical end of the spectrum and thus represent an over-correction from
the status quo. Other alternatives include: a) eliminating a train b)
shortening release cycles (possibly to days or hours) c) shipping frontend
(omni.ja and friends) updates separately from platform/libxul updates.
There's also the concern that system add-ons only solve the narrower
problem of shipping frontend components. Wouldn't time be better invested
in shipping all of Firefox faster? (I understand the post says that
modularity is the reason why you believe we can't go faster. But I get the
vibe from elsewhere that some feel packaging and shipping are significant
reasons.)

6) The emphasis on modularity and module boundaries in this post resembles
the latest trend in cloud software: microservices. I'll let readers
complete the analogy (there are both pros and cons).

Look, I understand the desire to "go faster" and deliver a better Firefox
to users faster. You know from what I've done with my time at Mozilla that
I care about developer ergonomics/productivity and contributor engagement.
Having worked most of my professional career in server-side software, I
understand the viewpoint that shipping monolithic software to clients seems
ancient and antiquated in comparison. That explains the desire to adopt
more of a server software mindset to delivering Firefox.

We should absolutely "go faster." We can "go faster" by adopting some
modern approaches to shipping. We can "go faster" by focusing on software
architectures that will make our paid staff and volunteer contributors more
productive. But I'll be honest, shipping code updates to core browser
features as discrete system add-ons that must conform to specific
[WebExtensions] APIs feels like it will cause more harm than good. I'd
rather focus on efficiently delivering a known, atomic unit of Firefox
(that doesn't rule out tweaking the behavior of code once it has shipped,
e.g. turning on a new feature N% at a time). Independently, we can improve
developer ergonomics for working on browser "internals." And I can think of
several ways we can do that don't involve significant architecture changes
or maintaining complicated and backwards-compatible API surfaces. Using a
typed programming language and/or static analysis to enforce cross-module
"good" behavior, improving support for IDEs and API discoverability,
writing JS as "modules first" with an emphasis on reuse and isolated
testing, and improving the overall build-edit-test-debug-review-land loop
come to mind as more important and far less effort from the perspective of
the Firefox front-end engineering team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20161007/2a45df10/attachment.html>