Thunderbird as a Web App revisited

Paul D. Fernhout pdfernhout at kurtz-fernhout.com
Sun Dec 13 18:18:02 UTC 2015


Andrew-

Thanks for the reply. I appreciate the time of people here who know so 
much about email and whose involvement no doubt would be essential to 
making a new Thunderbird concept that really works well.  I've started a 
week-long sprint to create a proof-of-concept of a Thunderbird 
Webapp/Server (which I really should not be doing, but that is another 
story). That way I can "show" rather than "tell". After that 
proof-of-concept is done, I hope you might be willing to look at a 
working (if very primitive) system and then we could expand on these 
discussions.https://github.com/pdfernhout/Twirlip2

In the meanwhile, here are some comments responding to points you 
raised, especially the practical difficulty of running a local server on 
a desktop when people expect seamless desktop apps.

== What Thunderbird does now

I think I have not explained the Thunderbird Server idea well enough and 
so caused some confusion, sorry.

Right now, Thunderbird Desktop has essentially a web browser internally 
(95% of its codebase) that talks to an email application (5% of the code 
at most) via custom internal function calls in C++. I'm just talking 
about essentially the same thing, except the email application runs 
under a local webserver and the client part is the standard Mozilla 
Firefox, and the protocols in between are JSON.

Those protocols are admittedly, as you point out, likely custom 
protocols. However, the traffic across them is at least inspectable and 
presumably documented. Those APIs are potentially callable by other 
clients -- something presumably in practice the current Thunderbird 
internal adhoc protocols defined by C++ functions are not.

So it seems to me like what I propose is still a step forward from the 
current situation? Even if it is not perfect? Still, involvement by 
people here in designing (or finding as with JMAP, thanks!) good 
protocols could help a lot. One issue I'm pondering is how to send 
search queries to get subsets of emails. I wonder if that is something 
JMAP could help with? So, leveraging that sort of thing might be a big 
win to save time, thanks again for pointing it out.

The only proviso I'd say is that I'm interested in integrated messaging 
-- so, email, IRC, RSS, whiteboards, note taking, and more, all 
together. A standard email protocol, even a better one like JMAP, might 
not provide everything needed to support a full feature set. But I guess 
I need to learn more about them.

The system will not be perfect at the start. I plan to put in some 
arbitrary protocols to do the minimum needed for a proof-of-concept. 
But, in an iterative way, I'm very open to someone who knows existing 
email protocols well (which is not me), saying, look, you could leverage 
that an have one less non-standard part in the application specifically 
here. That would be valuable advice.

== Not a regular email server, but it could be

Such a server could of course also support direct IMAP, POP3, or JMAP 
access from existing clients perhaps as plugins -- but that is not the 
main intent. Actually, until you mentioned it, I had not even thought of 
doing that. Thanks! :-)

== Local servers are just another way to modularize software

The confusion may be in my choice of the word "server". Is there a 
better one to use here? Maybe I should say "Thunderbird Webapp" instead?

People in general are just not used to running local servers. I've 
gotten used to it because I'm using Node.js more and more for all sorts 
of things, including just parsing data and transforming it. You are 
right to point to the added complexity though.

== Using a server is the only practical way to get Firefox to work well

But the reason I feel that is the way forward anyway (for Thunderbird as 
a Mozilla project at least) is because Firefox's support for local data 
is a problematical afterthought (as with the ignored Firefox issue I 
posted about that 1.5 years ago mentioned at the beginning of the 
manifesto). To get Firefox to work predictably as far as same-origin 
policy security restrictions, you tend to have to use Firefox to surf to 
a live website (even if it is a locally hosted website). I hope that 
situation improves through a deep rethinking by Firefox of what same 
origin policy means for local applications and local files, and such a 
Thunderbird Webapp project might drive such improvements, but that is 
the current state as I've experienced it -- using Firefox without a 
webserver involved is problematical (which is sad to me).

== Don't look into the Electron beam with other eye :-)

In the long term, hopefully Firefox will provide better support for 
embedding apps in the way you can do with Electron/Chromium like 
Automattic just did for WordPress/Calypso to run as a desktop app.

Now I could just suggest until then the Thunderbird Server project just 
do the same thing as Automattic did and run Thunderbird Server with 
Node.js bundled together with Chromium via Electron for easy install as 
a standard-looking desktop app -- "but that would be wrong". :-)

So, please don't look here, whatever you do: :-)
http://electron.atom.io/
"Build cross platform desktop apps with web technologies ... Use HTML, 
CSS, and JavaScript with Chromium and Node.js to build your app."

== Mozilla ignored a crucial need and now everyone pays the price

More seriously, as I pointed out in a post to Mozilla Governance:
http://pdfernhout.net/thunderbirds-are-grow-manifesto.html
"Beyond the value of such a communications platform itself, if Mozilla 
had prioritized this need to improve Thunderbird four years ago, Mozilla 
might have improved the Firefox platform in ways that helped keep up its 
market share up because Thunderbird was showing a need that became 
Electron and Phonegap and other similar software. Then 
open-source-friendly companies like Automattic might not be turning to 
Electron/Chromium for creating a desktop WordPress/Calypso app instead 
of using the Firefox platform. In fact, if Mozilla has done such a 
thing, is it possible that much of the time and money and attention that 
has gone into WhatsApp, Viber, Line, etc. might have gone into an 
enhanced Mozilla Thunderbird and Firefox instead? Maybe then Firefox 
market's share and Mozilla's revenues might have even expanded rather 
than contracted? Instead, by ignoring the needs of Thunderbird and the 
internet users it serves, by not trying to grow in that area, Mozilla 
missed a huge business opportunity. ... Frankly, as I see it, if you 
look at the adoption trends, and consider the rapid rise of Slack, 
between Firefox and Thunderbird, an expanded Thunderbird is actually the 
more viable product concept long-term. :-) But I am sorry to have to say 
that as a long-time Firefox user. :-( ..."

That post is still in the moderation queue and I wonder if it will see 
the light of day. Probably should never post stuff at 5:30am after 
staying up all night writing it. I added the text from it to the 
manifesto near the end though.

== Benefits of the web server approach

A big benefit to of the webserver approach is the Thunderbird project 
could stop maintaining a second copy of Firefox it uses internally 
because it can just use regular Mozilla Firefox on the front end 
(removing 95% of the Thunderbird codebase and much drudgery, patching, 
and security issues). Switching from C++ to JavaScript for the entire 
app could also reduce memory bugs and security issues. But that is not 
the only benefit.

With the webserver approach, you could potentially also access your 
email (and perhaps other communications like IRC chats, RSS feeds, 
notes, and more) from any device on your local network (including 
phones, tablets, watches). You could setup custom networked devices like 
a lamp that lights up when you get a message from someone special.

It would also be possible to access your email outside your network 
depending on your network configuration (although that is a risk).

Those are things you can't easily do with Thunderbird Desktop (unless 
you do virtual screensharing over the network perhaps, but that is 
tedious and impractical for a phone).

For me, I use POP3 and not IMAP. I have Thunderbird delete email from 
the server (after a short delay in case of a hard disk crash). I don't 
consider myself having an existing "email server" in the sense that I 
see our email server just as a relay point, not something that stores 
email. I can imagine someone who uses an IMAP (or JMAP) server regularly 
might have different expectations though.

Some people would not use an IMAP server at a remote host in the way it 
was intended for privacy/legal reasons, because files more than six 
months old become easily legally accessible in the USA on servers. For 
example:
http://www.mcclatchydc.com/news/politics-government/congress/article24779989.html
"If you’ve been remiss in cleaning out your email in-box, here’s some 
incentive: The federal government can read any emails that are more than 
six months old without a warrant. Little known to most Americans, 
ambiguous language in a communications law passed in 1986 extends Fourth 
Amendment protections against unreasonable search and seizure only to 
electronic communications sent or received fewer than 180 days ago. The 
language, known as the “180-day rule,” allows government officials to 
treat any emails, text messages or documents stored on remote servers – 
popularly known as the cloud – as “abandoned” and therefore accessible 
using administrative subpoena power, a tactic that critics say 
circumvents due process."

== Why this should be a Mozilla priority

People at Mozilla believe in web technologies. That part I loved about 
Firefox OS and its approach -- as even knowing how to write Android apps 
I'd rather work on cross-platform webapps.

Given that belief, making email accessible over the web, but stored 
locally and accessed securely to increase privacy, seems to me like a 
no-brainer for Mozilla.

Of course, one may then discuss priorities, but given 70% of user 
accounts are either email or IM, by one estimate from The Radicati Group 
I cite in that latest governance email, it seems supporting such 
internet usages of email and IM should potentially have even higher 
priority at Mozilla than improving access to social media accounts (the 
other 30%). But Mozilla's current funding priorities, especially 
defunding Thunderbird development, obvious do not reflect those 
statistics. I just don't understand that. :-(

(Anyway, back to coding...)

--Paul Fernhout
http://www.pdfernhout.net/
====
The biggest challenge of the 21st century is the irony of technologies 
of abundance in the hands of those still thinking in terms of scarcity.

On 12/13/15 12:27 AM, Andrew Sutherland wrote:
> Disclaimer: I am client-side biased since I work on a purely client-side
> HTML/JS email app (the Firefox OS Gaia Mail app).
>
> Especially given the points made about local data, I was a little bit
> surprised about the emphasis placed on involving an (additional) server.
>   One of my concerns about mailpile has been that it involves an
> additional server that is not the user's actual mail server.  This
> introduces logistical and cost issues that complicate things and beg the
> question of whether an additional server is needed.
>
> Which is not to say there aren't things that are best done on the
> server.  But if they are best done on a server, why not the one the user
> already uses (/ switches to from their previous provider)?  It's easy to
> technically hand-wave that they can be co-located or an existing mail
> server integrated or written, etc., but these are not trivial issues.
> And they especially matter because email is largely a cost-conscious
> domain.  Most users do not seem willing to pay for high quality service,
> and mail providers are not rushing to provide it for those users.
> Things that seem fundamental like IMAP support or valid TLS certificates
> are far from dominant.
>
> Traditionally, the rationale/argument against this is that one can't
> depend on all servers to implement the features one needs or implement
> them well so one needs to implement them oneself.  This, combined with
> Thunderbird's POP3 support and general assumption of a desktop-class
> machine with bountiful storage, is one of the reasons why Thunderbird
> efforts in 3.0 and 3.1 were focused on local search instead of
> improvements to search-on-server.
>
> But things are changing!  Fastmail, which uses and actively (helps)
> develop the excellent open source Cyrus IMAP server has been actively
> proposing a new protocol, JMAP (http://jmap.io/).  This provides a sane
> and lovely API that guarantees many things that have never been safe to
> assume about an IMAP server (unless you saw the post-login CAPABILITY
> string).
>
> Now, this does not mean that we can assume mail providers out there will
> upgrade to the latest Cyrus, but it does mean:
> - There's a production-grade open source server that exists and is able
> to run sufficiently cost-efficiently that Fastmail can make money.  If
> users want this nice server, there is already at least one provider they
> can feasibly pay to get it.  If they want to run it themselves, they can
> do that too.  The introduction of an additional server presupposes that
> they are willing to do one of these things.
> - There's an actively developed open source mail server trying to make
> things better and that is actively invested in the needs of its users.
> (Note that I'm not saying others don't exist, I'm just not aware of any
> others interested in JMAP adoption at this time.)
> - Using the wants-to-be-a-standard-and-looks-like-it's-going-to-be JMAP
> for client-server communication is way better for the email ecosystem
> than a Thunderbird web app/server implementing what amounts to de-facto
> proprietary APIs.  (Obviously, the Thunderbird API need not be
> proprietary, but standardization requires adoption/multiple
> implementations and if everyone is just making up their own APIs and
> ignoring other standardization efforts then it's still proprietary.)
>
> Summarizing:
> - I think anything that aspires to be a Thunderbird successor should be
> client-only with no proprietary server-specific bits.
> - Server-specific needs can and will exist, but they should be addressed
> by working on/with IMAP/JMAP/other standards.  (And in many cases most
> needs can probably be addressed by using ANNOTATE or something like
> that, although it looks like JMAP does not require ANNOTATE support but
> will allow it.  I need to do more research there.)
>
> Andrew
> _______________________________________________
> tb-planning mailing list
> tb-planning at mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning
>



More information about the tb-planning mailing list