Followup: modularity, WebExtensions, and going faster

Sat Oct 8 05:37:01 UTC 2016

On Sat, Oct 8, 2016 at 12:12 AM, Robert Helmer <rhelmer at mozilla.com> wrote:

> On Fri, Oct 7, 2016 at 2:48 PM, Gregory Szorc <gps at mozilla.com> wrote:
> > 3) If we're talking about a significant change of architecture to the
> > Firefox frontend and the high costs associated with that change, I'll
> throw
> > out an alternative idea: (re)write the frontend in a typed language like
> > Rust. I argue that a typed language - and Rust especially - would deliver
> > "well-defined, documented, and enforced module boundaries or interfaces"
> > better than JavaScript ever will.
>
>
> +1. I think we should probably stick with HTML/JS/CSS for basic UI
> stuff like layout, setting up button events, DOM manipulation etc, but
> a lot of the code we have in JSMs right now would be much safer to
> implement in Rust, and most likely faster.
>
>
> > 4) I have significant concerns about the concept of system add-ons.
> > Currently, the composition of Firefox (in terms of code) is essentially
> > deterministic from the commit in mozilla-central it was built from. We
> know
> > exactly what Firefox code is running (OK, /should/ be running) on users'
> > machines. System add-ons fundamentally change that from ~1 combination of
> > software components to N, where N could be absurdly large (10^3 or
> larger).
> > It is hard enough to track down intermittent failures in test automation
> > with a well-defined software configuration. System add-ons introduce
> whole
> > new classes of bugs related to how different versions of add-ons interact
> > with each other. They require more and more complicated testing
> > infrastructure to test various combinations of add-ons. This makes
> assuring
> > quality harder. System add-ons seem like they add cost to the release
> > process and undermine stability (even if they may make writing code
> easier -
> > but software development is about a whole lifecycle, not just writing the
> > code).
>
>
> I think we're largely in agreement overall (see my earlier post on
> this thread), but just to help assuage any concerns about our current
> use of system add-ons - we require all extensions that ship with
> Firefox to be in-tree, and any out-of-cycle updates should be landed
> on the appropriate branches. So, they live alongside other code in
> Firefox, are tested as part of the usual automation and QA, etc.
>
> We do have the ability to push arbitrary updates, but we only use
> those for hotfix-style activities, similar to the current hotfix.
>
> Any out-of-cycle updates pushed "expire" when the next regular
> application update happens.
>

Thanks, Robert. This does significantly assuage my concerns about the
system add-ons model, especially around things becoming out of sync. In my
defense, I was thrown off by the line in the blog post about potentially
allowing add-ons to replace browser components. Assuming there is no
slippery slope argument to be made here, then the conversation is
essentially about how we should architect software within mozilla-central.
In that case, I refer to rnewman's post on the topic, which was terrific.

FWIW, part of my unease about shipping core features as add-ons - possibly
independently from major updates - is reaffirmed by an experience I had
several weeks ago. One of our Test Pilot extensions broke enter/return in
the awesomebar and I was literally unable to open web pages. My Nightly
install was essentially bricked. I can tolerate Nightly crashing
excessively every now and then - it is a daily build after all. But in my
5+ years of using Nightly, I don't recall having a Mozilla-maintained
frontend feature fundamentally breaking the browser like that. I suppose I
could chalk it up to chance and the intrinsic relative instability of
Nightly. Yet at the same time, I know that we don't put Test Pilot
extensions through the same [absurdly comprehensive] test suite that CI
runs for commits to mozilla-central. I'm confident that if we did (or if we
developed Test Pilot features in mozilla-central like we do for all other
browser features), this breakage would have been detected and we never
would have shipped. So when we start talking about developing and shipping
features as add-ons outside the existing monorepo and CI infrastructure
built around it or allowing core components to be swapped, I get really
anxious that product quality and stability will suffer.

>
>
> > 5) I challenge the necessity that system add-ons or WebExtensions are
> > necessary to "go faster." There are a host of changes we could make to
> the
> > release process to "go faster." System add-ons feel like they are on the
> > more radical end of the spectrum and thus represent an over-correction
> from
> > the status quo. Other alternatives include: a) eliminating a train b)
> > shortening release cycles (possibly to days or hours) c) shipping
> frontend
> > (omni.ja and friends) updates separately from platform/libxul updates.
> > There's also the concern that system add-ons only solve the narrower
> problem
> > of shipping frontend components. Wouldn't time be better invested in
> > shipping all of Firefox faster? (I understand the post says that
> modularity
> > is the reason why you believe we can't go faster. But I get the vibe from
> > elsewhere that some feel packaging and shipping are significant reasons.)
> >
> > 6) The emphasis on modularity and module boundaries in this post
> resembles
> > the latest trend in cloud software: microservices. I'll let readers
> complete
> > the analogy (there are both pros and cons).
> >
> > Look, I understand the desire to "go faster" and deliver a better
> Firefox to
> > users faster. You know from what I've done with my time at Mozilla that I
> > care about developer ergonomics/productivity and contributor engagement.
> > Having worked most of my professional career in server-side software, I
> > understand the viewpoint that shipping monolithic software to clients
> seems
> > ancient and antiquated in comparison. That explains the desire to adopt
> more
> > of a server software mindset to delivering Firefox.
> >
> > We should absolutely "go faster." We can "go faster" by adopting some
> modern
> > approaches to shipping. We can "go faster" by focusing on software
> > architectures that will make our paid staff and volunteer contributors
> more
> > productive. But I'll be honest, shipping code updates to core browser
> > features as discrete system add-ons that must conform to specific
> > [WebExtensions] APIs feels like it will cause more harm than good. I'd
> > rather focus on efficiently delivering a known, atomic unit of Firefox
> (that
> > doesn't rule out tweaking the behavior of code once it has shipped, e.g.
> > turning on a new feature N% at a time). Independently, we can improve
> > developer ergonomics for working on browser "internals." And I can think
> of
> > several ways we can do that don't involve significant architecture
> changes
> > or maintaining complicated and backwards-compatible API surfaces. Using a
> > typed programming language and/or static analysis to enforce cross-module
> > "good" behavior, improving support for IDEs and API discoverability,
> writing
> > JS as "modules first" with an emphasis on reuse and isolated testing, and
> > improving the overall build-edit-test-debug-review-land loop come to
> mind as
> > more important and far less effort from the perspective of the Firefox
> > front-end engineering team.
>
>
> I agree. I think our problem is that hacking on Firefox is
> exceptionally hard, and testing+shipping it is too. I don't think
> either of things are hard because there aren't good enough boundaries
> between different components.
>
> Pushing updates quickly to just omni.jar and allowing restartless
> updates is basically equivalent to moving features to extensions and
> updating them that way. It just so happens that today it's easier for
> a new team to get going by hacking on an add-on, it's faster and safer
> for us to ship these standalone rather than update the whole of
> Firefox, and extensions already provide a way to do restartless
> install+update.
>
> I don't think it follows that therefore everyone must develop on
> extensions and all updates should be done the way extensions are. We
> can and should make Firefox easier to hack on and should make shipping
> faster + safer.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/firefox-dev/attachments/20161008/3cdc4998/attachment.html>