[Go Faster] shipping quality code faster

Robert Helmer rhelmer at mozilla.com
Mon Sep 28 21:12:55 UTC 2015


On Mon, Sep 28, 2015 at 12:10 PM, Chris Hofmann <chofmann at mozilla.com> wrote:
>
>
> On Mon, Sep 28, 2015 at 10:54 AM, Robert Helmer <rhelmer at mozilla.com> wrote:
>>
>> I don't have all the answers here, but when I said "autonomous" I more
>> had in mind partners like Pocket, who are already working on their
>> own, with their own processes, and giving us code dumps.
>
>
> After a group like the pocket developers give us a "code dump"
> what happens from there.  Would they work autonomously on all the
> rest of the stages toward release of promotion to press and users, or
> at some point do they start working less autonomously and more integrated
> into our defined release process?


As I mentioned before, I certainly don't have all the answers here,
but happy to share my opinions (I've got lots!) I think we're getting
bit off-topic for this thread though so I've changed the subject.

I would argue that anything that ships as "Firefox" needs to be
reviewed and put through our standard security, QA and post-release
processes (crash analysis, telemetry and so on). We should continue to
invest in and improve these processes.

One argument here is if the overall process *needs* to take 6 weeks
from code landing in-tree to getting into user's hands, and (if so)
are add-ons the right vehicle to deliver this.


>> My impression
>> is that we're going to have more partner integrations in the future
>> and we want to try to improve the situation here.
>
>
> It sounds like a good impression to confirm, and understand more
> about how the interaction and process toward pushing out bits to
> each of the testing and release channels will work.


To continue using Pocket as an example - that ended up jumping ahead
of the usual train model to release on time, right? I've spoken to a
few people involved in the process (mostly around the "how" not the
"why") but I don't know the whole story.

Is the right thing to do here to push back and say 6 weeks is as fast
as we can go? Or to provide a way to ship standard Firefox updates in
less time, rather than splitting these sorts of features out to
add-ons?


>> The Hello team largely operates within normal Mozilla processes right
>> now as you describe. The release process is the only thing I imagine
>> them having more autonomy around.
>>
>
> By release process to you mean pushing the button on making the bits
> available,
> or some other pieces closing related to that process?  It would be good
> have a common defintion of that term and where system addons
> lend benefit.
>
> I'm hopeful that we won't be by passing important
> testing, security evaluations, crash measurement, and all the things that
> we do at the expense time in an effort to raise or maintain quality.
> There are even more checks like static analysis of code, and response
> *before* landing, that suggested by roc and others over on the thread
> Sylvestre started on dev-platform.
>
> This kind of increased scrutiny of code could be interpreted as
> reducing the amount of autonomy of a developer or system addon
> development team might have if there goal is to just get their addon
> shipped, but it would be good to get this definition nailed down.


I am certainly not suggesting we skip things like QA, security
evaluations and crash measurement. While we should be able to do
things like ship features to a subset of users and get more real-time
feedback on how things are going before rolling out further, we still
do need to review and evaluate and test the product as a whole before
we even begin the update process.

One interesting side note about static analysis of code - while we
can't do true static analysis for a language like JS, addons.m.o does
have a fairly strict validator that must pass before add-ons can be
signed. We don't have anything this tough for in-tree code right now.

Not that I'd object to all in-tree code needing to pass strict
validation! In fact that's something I am very interested in, the fact
that we use C preprocessor instructions in our XUL/JS/CSS makes this
hard to use standard tools right now but there's progress on that
front already. The Hello codebase and some parts of mozilla-central at
least use eslint now, but we could be scanning for security
vulnerabilities and other bugs as well.

I think that automated checks (including static analysis if
applicable) for code before it even gets to a reviewer is something we
should be doing. The earlier we catch bugs in the development process,
the better.


> To reduce chemspills and user frustration we are going to have to find
> ways to raise the quality bar and allow for more testing and this might be
> at odds
> with "developer autonomy"  and also "shipping faster" so I think its a
> worthwhile
> thing to define clearly since there could be some tension there between
> requirements.


We want to both ship a higher quality product, and we want to ship it
faster. I don't think we need to trade these off against each other,
but there's always a cost.

In this case we're talking about not riding the trains for a full 6
weeks, mitigated by the fact that the changes are compartmentalized
inside an add-on. The more we can enforce this boundary the better.

We certainly achieve do this *without* add-ons, which reduces
complexity in some areas. The work we've done so far extends the
already-existing and well-tested add-on code in fairly simple ways,
and we are planning on using our standard build/release and update
server.

Thinking ahead a little, evolving these to be web extensions would
allow us to have a much stricter boundary between "application" and
"extension" code, which I think would give us even more assurance that
e.g. Hello couldn't break something in other areas of the browser.

Another avenue, which we should be pursuing as well, is to make more
of our content remotely hosted, for instance all the content bits of
Hello are just standard HTML/JS/CSS. This would reduce the need to
even push changes to the add-on, unless there was a need to touch the
chrome bits.

There's already an experiment ongoing with the about:newtab page using
remotely hosted content, I'd like to see the result of that before
proposing beginning any work there.


More information about the Gofaster mailing list