Proposal to start a new implementation of Thunderbird based on web technologies

Ben Bucksch ben.bucksch at beonex.com
Tue Mar 28 16:08:23 UTC 2017


Hey Paul.

Paul D. Fernhout wrote on 28.03.2017 04:46:
> This is a great proposal! Some ideas below to build on it.

Thank you for the responses :)

> ## Minimal Viable Product
>
> I completely agree the complete conversion effort in total will take
> years (especially to move the plugin ecosystem over). But, to maintain
> significant interest in the effort, I'd suggest aspiring for having a
> minimally usable system to view and send email

Completely agreed! I subscribe to the same idea.

That's why I defined 3 major milestones. I want to set expectations. It 
would be highly unrealistic to have feature parity with Thunderbird in 
the first release. But we can produce a product that will be usable to 
some parts of the userbase much earlier.

> as plain text within a month of starting (as in, one person full-time for a month).

This, however, is a little ambitious and can be very detrimental. 
There's a lot of groundwork to lay. With design and framework, it's 
harmful to get started without it and hack it first and try to use a 
better design later, because it's not realistically possible anymore. 
You'll have to change everything again. Realistically, it will not be 
done. So, whatever design you use at the start is what you will always 
have. Thunderbird is a good example of this. It still, up to this day, 
has the very same design and architecture that it had 18 years ago. 
There are clear shortcomings, with clear disadvantages (e.g. unclear API 
separation, module dependency that looks like a cat had her way with a 
ball of wool, no clear extension hooks, thus hard to hack), but nobody 
can solve them or has the will and manpower to solve them. 
MozillaMessaging wanted to, and tried, but couldn't, despite 6-10 people 
on payroll.

The framework and APIs need to be designed well from the start. Likewise 
the widget set. That will massively influence the quality of the 
outcome, and the joy it is to hack it.

For example, I want to create consistent and clear APIs between backend 
and UI, and between different UI parts.

> ## Thunderbird interoperation from the start

I think we'll support IMAP from the start. I already have a partial, 
read-only implementation of IMAP and POP3 in JavaScript. Of course it's 
just a start, but it shows it's feasible.

IMAP is inherently built to support multiple clients on the same account 
(preferably not at the same time, although it's possible in theory).

> Yes, anyone who does the above is living dangerously as they put their
> real email at risk and so needs to make frequent backups.

yup :)

> ## Mobile from the start

I want the new implementation to run on smart phones, yes. Using 
Cordova. Obviously, we need a different UI layout, but fundamentally, it 
should be possible.

That smartphone UI wouldn't be something I write from the start, but I 
want to architecturally enable it from the start.

> ## Analytic Visualizations from the start

This is going directly against your MVP above :). First we need to read 
and send email.

I totally want visualizations. I really hope for it. But that's 
something for year 3 or 4. That's the icing on the cake. You first need 
to lay the groundwork.

If you (or anybody else) have ideas for concrete visualizations, could 
you please post them somewhere? I am very much looking for ideas in that 
area. (I am looking at asutherland's glodastrope already.)

> ## Technology stack
>
> I'd suggest using TypeScript, Mithril.js, Tachyons, and
> Node.js for the base technologies -- probably with Mocha/Chai for
> testing. Later we can make a more stand-alone desktop version with
> Electron or such.

Thunderbird is going to be a desktop application from the start. The 
most fundamental reason is what we need TCP sockets for IMAP and POP3, 
and read/write local folders, using mbox, maildir or other formats.

There are several email clients already that go the web server proxy 
way, e.g. Nylon N1. That's also why I don't use any of them. I want a 
direct replacement for Thunderbird.

Using TypeScript is something we could consider.
We'll most likely use node.js, in some form or other - if nothing else, 
then for the npm module installation.
I'll take a look at the other libraries.

If any one of you has experiences with good JS libraries, that's 
probably useful. I want to be easy on them, because the more JS code we 
import, the slower the app will be, but if a lib is small and provides a 
huge benefit, that's helpful.

> We could use socket.io to push notifications for
> changes in real or virtual folders and then use Mithril.js (a vdom
> library) to quickly rerender changed data the client UI requests (or is
> pushed).

Mithril.js looks very similar to the mini-lib that I've been writing for 
myself <http://benbucksch.github.io/trex/util.js>. It looks nice, it's 
worth considering.

I want to put observable lists everywhere in the API. Virtual folders 
are just one application. I've already written the library 
https://github.com/benbucksch/jscollection I am not aware of anything 
else that can do dynamically observable operators.

> ## A previous step forward by me
>
> Here is my past attempt in the direction of a web-technology-based
> Thunderbird Server (with some help by David Krings for design and
> testing) -- but I ended up taking a full-time+ paying job doing UI work
> for a SV company soon after starting that effort, so not much time for
> moving it further along, sadly.

Too bad, would have been nice to have you helping with this endeavor.

> https://github.com/pdfernhout/Twirlip2

> A key useful thing I developed there (improving on something first
> developed for another project) was displaying HTML reasonably safely
> using Mithril (crossing fingers).
> https://github.com/pdfernhout/Twirlip2/blob/master/source/webapp/ts/sanitizeHTML.ts

Lovely. That's the same fundamental approach that I also used in the 
"HTML sanitizer" used in Thunderbird ("Message | View | As Simple HTML").

For rendering messages, I want to use a sanitizer like that, plus 
additionally an <iframe> sandbox (different domain) on the browser 
level. When it comes to security, I believe in multi-layered defenses. 
I've seen too much ;).

> [1] Ultimately the safest way to handle displaying full HTML might be to
> have every email or RSS feed be displayed its own subdomain served by
> the server using the item's message ID...

Exactly.

> http://twirlip.com:9000/twirlip.html

Nice! to have a running demo on the web :)

> ## Learning from my mistakes
>
> One of the (many) too ambitious parts of the above effort was to use a
> different storage backend.

MVP :)

> "The biggest challenge of the 21st century is the irony of technologies
> of abundance in the hands of those still thinking in terms of scarcity."

Surely true when it comes to contributors :-) . The more inviting and 
exciting we are, the more will come.


Thanks for this qualified response! Too bad you can't help us with the 
implementation.

Ben


More information about the tb-planning mailing list