Why we need Gecko updates

Andrew Sutherland asutherland at asutherland.org
Fri Dec 11 06:44:18 UTC 2015

On Thu, Dec 10, 2015, at 06:06 PM, R Kent James wrote:
> In my dreams there is a grand coalition of several key players in the 
> open-source email client world, that would all work together to make 
> something that could be truly competitive to the mega players in this 
> space. Thunderbird is part of that dream team, as well as Gaia email, 
> Postbox, and perhaps other newer players like N1 or others.

The major difficulty with banding together is that many of the existing
open source clients have all made and are all committed to mutually
incompatible architectural and implementation decisions.  Mailpile and
Nylas use Python based intermediary servers (like the Mozilla Messaging
"raindrop" experiment).  Thunderbird, Postbox, and Gaia Email are client
side without intermediary servers.  Much of Thunderbird and Postbox are
C++, while Gaia email is pure JS, etc.

There needs to be overlap to find synergies/efficiencies/etc., and in
many of these cases there is little overlap.

Thunderbird and Gaia email have already found some overlap in terms of
low level details.  On gaia mail's conversations development branch,
we're using Joshua Cranmer's excellent work on JSMime for MIME parsing. 
Gaia email has also been able to make use of and contribute to the many
excellent http://emailjs.org/ libraries authored and maintained by
Andris Reinman, Felix Hammerl, and Tankred Hase of the (regrettably now
defunct) whiteout.io email client.

> I am pushing very hard to get the JsAccount work landed in TB 45, so 
> that JS-based backends could hook into TB. The Gaia Active Sync 
> implementation would be a great demo of that, perhaps I'll look into 
> that.

ActiveSync is a frustrating, limited protocol in its normal usage-mode
that we implemented to support hotmail.com/outlook.com prior to their
(buggy but improving in quality) IMAP support.  To a lesser extent, we
also wanted to support Exchange servers, but it's hard to know if we
succeeded in that because we have no telemetry and have no test accounts
on old exchange servers, only new ones.  I would advise against
supporting it!  But if you really want to support ActiveSync, the gaia
libs at https://github.com/mozilla-b2g/jsas and
https://github.com/mozilla-b2g/jswbxml/ and perhaps our higher level use
of them at
would probably useful to build on.

I think potentially a more interesting account type would be support for
"backfilling" mailing lists by ingesting their archives.  Much of this
would likely resemble heuristics/rules for figuring out where the
archives are for the various mailing list managers informed by list-*
headers and then using an appropriate mbox-like parser.  This is again
somewhat more low level, but I think makes a perfect area of overlap
between Thunderbird between gaia mail and other JS mail clients.  I
would love to help contribute to something like this, and hopefully
others would too.

> If you have other ideas of ways we could try to work together, I 
> would love to hear it. I know you have been thinking about expanding the 
> Gaia Email work into a desktop client. If you do, how can we work 
> together instead of apart?

In the immediate term, I think our area of overlap continues to be JS
implementations of protocols.

Longer term and at a higher level, it's tricky.

My vision in all of my work on Thunderbird was to use gloda as a
normalizing abstraction layer on top of the existing folder/message
representations.  It added an understanding of cross-folder
conversations and other nice things like the potential to handle people
having multiple email addresses, etc.

The idea was to build new UI stuff on this, and once we had the UI
sitting on top of that and decoupled from the existing highly-coupled
folder/message storage mechanism, we could ideally overhaul the
underlying message stores, etc.  (The big problem with all of this is
captured in the phrase "new UI".  New UI of such a magnitude arguably
ends up being a new product and not the existing Thunderbird people know
and love.)

My underlying goal for the gaia email app backend has always been to
produce a back-end capable enough to meet the needs of a powerful
desktop mail app as well as its immediate goal of being a powerful phone
mail app.  And to do so informed by what I learned from my efforts on
posterity, core Thunderbird, gloda, junius/raindrop, and deuxdrop.

It is my hope that sometime, not so long from now, I can make a post
here where I suggest that the gaia mail backend has reached that
threshold and that we can all now try and create an HTML-based UI that
is able to re-create the essence of what Thunderbird users love.

I haven't been shouting this goal from the rooftops because it has been
and continues to be a hope and not a guarantee.  We've all seen repeated
claims of "reinventing email"/etc. and I'm very sensitive to creating
harmful stop energy.  Thunderbird has been and continues to be a viable
product despite its technical debt issues, and I wouldn't want to
discourage people from working on it or coming up with solutions I was
not wise enough to see.  Additionally, I knew this would be a long slog
(though it's been much longer than I envisioned to get to where things
are now) and could not make guarantees about many things: continued
funding, that I wouldn't burn out and become a mime, etc.

If you want to shoot for maximum synergy with gaia mail and do it
starting now, here are some good options I see:

- I can use help on the existing gaia email conversations refactor.  The
pre-reqs on this based on current velocity and complexity are probably
someone with 10 hours/week to spend and a fairly good understanding of
the email problem domain, basic database transaction semantics, and a
reasonable understanding of ES6 (destructuring, generators).  A good
litmus test is probably for a potential contributor to look at the gloda
code and see if they understand what it's generally doing and can resist
cursing too much while reading the code.  (You made some very
significant contributions to the batching mechanism and I don't think
there was any cursing in the bugzilla comments and so are pre-qualified!

- Assume the gaia email backend will get there and try and help distill
down what makes Thunderbird what it is so we can move faster when it
gets there (which it may not).  Create UX specs, mock-ups, prototypes,
add telemetry to Thunderbird to understand better how people use it,
etc.  For example, it's my (arbitrary, not-based-on-data) theory that
people like the "tactile" feel of the Thunderbird thread-pane that lets
you scrub through messages.  I think that its many columns, support for
sorting on all columns, and high performance at scrolling and the fact
that it shows you *all* the messages helps give you confidence that your
messages are there and that you can find what you're looking for, even
if you have to scroll for a long time.  In contrast, paginated and
search-biased interfaces like gmail may leave you feeling like your mail
is a haystack and all you can do is walk around the periphery and plunge
your hand in randomly and hope you find that needle.

The problem is that most of this is not useful to Thunderbird itself
right now.  And similarly, while the JsAccounts you implement could be
useful in a hypothetical gaia-mail-backend future, the JsAccount effort
itself is probably not since I would assume it uses existing Thunderbird
folder/message semantics.  Indeed, most efforts on Thunderbird that are
not pure JS/HTML and involve XUL or anything starting with nsI* are
probably not.  And writing that, that seems 1) jerky, 2) stop energy, 3)
I kinda really want to delete this paragraph but based on your comments
it seems like I have been under-communicating my own personal vision so
I won't.  But since I'm not deleting this, I absolutely want to convey
how tremendous your efforts on Thunderbird technically and
organizationally have been from my perspective.  While you have
different technical opinions than I do, I absolutely have confidence in
your technical skills and am confident your decisions are based on
bad-ass expertise.  (And with the rest of Thunderbird assumed, JsAccount
does seem like a very wise implementation choice.)

Hopefully this message is somewhat helpful in clarifying the state of
things re: gaia mail even if it doesn't provide immediate hope for the
collaboration that we all want.  Unfortunately, I should probably also
call out that Firefox OS is currently in a state of priority flux and
while I personally hope current MoCo engineering resources on the gaia
mail app continue at current levels of effort or greater, I can't
guarantee that they will.  What I can say with high confidence is that I
personally will continue to hack on the gaia mail app backend in my
spare time even if I am no longer paid to, fwiw.


More information about the tb-planning mailing list