<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Now is perhaps as good a time as any other to start thinking
about technical hurdles to preparing the modules of
Thunderbird.next (or whatever name people wish to call it). Here
are some things that we are likely to need to start to come to a
consensus on before getting any serious headway in coding:</p>
<h2>JS support and platforms<br>
</h2>
<p>Obviously, we need a minimum baseline to support all the
platforms that we wish to support. Something like
<a class="moz-txt-link-rfc2396E" href="https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code"><https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code></a>
is a natural way to display and organize this information--you
make a list of new and upcoming features, what the minimum version
of the various platforms is to support them, and you slowly move
the bar higher as you require newer versions. It also lets you see
what requiring newer versions gets you in terms of being able to
improve baseline support.</p>
<p>For a starting point, I think it makes sense to start with
baseline of ES6, perhaps with async/await support. There is one
major problem though: ES6 modules, while technically supported by
all JS engines at this point, lack a specification for how to
actually locate and load modules, with the current status that
they're pretty much unsupported by everyone at this point. While I
think everyone agrees that it is the eventual solution, it looks
like it will be at least a year before there's usable support in
any environment. And, as time has told in the past before, it's
best to not expect an "imminent implementation" until there is a
preview version of it actually shipping.</p>
<h2>Testing<br>
</h2>
<p>Especially since JS lacks the ability to let the compiler catch
common errors, a thorough test suite is absolutely necessary, and
needs to be considered from day one. There's effectively three
kinds of tests: low-level unit tests that test an individual
component (e.g., IMAP protocol code) in isolation of a larger
system, a high-level UI testing that drives the application from
the top (à la Mozmill today), and system integration tests that
might make sure that things like mailto: links work correctly or
tests code that relies on changing settings that actually affect
the computer (e.g., GSSAPI or system keyring issues). We don't
really have any tests of the latter kind today, and it's worth
noting that it is painful to test that kind of code when the
user's system might be acted upon. I've suggested in the past
using Docker to help with this, and it looks like there is some
sort of support for now running Windows guests in a Docker
container, although it's not clear if such support will involve
monetary outlays in the future. As for UI testing, I have no real
experience with anything here and so have no real comments to
give. For low-level unit tests, I've come to like the mocha tests
(although in TDD style, since I prefer suite/test over describe/it
for some reason--note that TDD is not the default) as a good
testing framework, and this seems to be a fairly common one in the
JS community.</p>
<p>Another related but completely different aspect of testing is
performance tests. Performance benchmarks are notoriously hard to
get right, but it's increasingly critical these days for major
applications (and Thunderbird certainly has a large enough
userbase to be a major application!). Especially since JS has a
not-entirely-undeserved reputation for being a slow language, I
think it is crucial that we get started with performance testing
very early, and we set strict guidelines for performance targets
(I use the example of 1M message folders or 10-20MB email messages
to guide my idea of what large things we might have to reasonably
deal with, and these definitely give high constraints for code).<br>
</p>
<h2>Library fragmentation and sharing code</h2>
<p>I don't know a good pithy name for this, but it's fairly clear I
expect that everyone thinks we should be dividing the
implementation into smaller modules/packages/etc. of some size. I
think it's a good idea to set guidelines for where we expect these
package boundaries to lie, and particularly on what we should do
with those kinds of functions that might feel too low-level to ask
for a dependency but crop up in many packages (base64 and email
address manipulation come to mind). A related issue is that we're
likely to have to polyfill some implementations into stuff in base
APIs. For example, we probably eventually need UTF-7 support in
text conversion, and NTLM and CRAM-MD5 require crypto algorithms
not exposed by WebCrypto (but might be exposed by other crypto
implementations).</p>
<h2>API consistency for platform-inconsistent stuff</h2>
<p>JS has a rather narrow standard library, which means a lot of key
APIs have no real standard way to reflect them. This means we're
going to have sit down and start figuring out common APIs to
represent what we need to bridge these differences--which may mean
adopting one API and polyfilling everybody else, or it might
involve writing our own API and polyfilling everybody. A brief
list of the things I can think of:</p>
<ul>
<li>FFI support (we'll need it for LDAP, OS integration, GSSAPI,
possibly things like S/MIME, import, PGP)</li>
<li>Streams: the performance targets I have require that things
like MIME decoding need to happen in some sort of I/O stream, so
this concept is going to run pretty deep in our code</li>
<li>Crypto</li>
<li>Sockets</li>
<li>Charset conversion</li>
<li>Database (I think we need ACID-compliant database, and I don't
think we want to roll our own here. Note too that "load it all
into memory" is not going to satisfy performance requirements
when dealing with large folders).<br>
</li>
<li>Filesystem</li>
<li>DNS, HTTP, SSL, other network support</li>
<li>Multicore (whether multithreaded, multiprocess, or both)</li>
<li>Modules (might as well include it in this list since ES6
modules isn't an immediately available option)</li>
</ul>
<h2>Boring infrastructure stuff that can be surprisingly
controversial</h2>
<p>Where do you host repositories? Issue tracking? Roadmaps? Build
instructions? Support? Documentation? Continuous integration?
Mailing lists? Updates? What license are you going to use? What
build system? Test infrastructure? Documentation tool? Linting?
Static analysis? Some of this stuff can be deferred or changed
after coding begins in earnest. Some of it can't.</p>
<p><br>
</p>
<p>There's probably more topics to think of that I haven't included
in this post, but this should definitely be a good starting point.<br>
</p>
<pre class="moz-signature" cols="72">--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist</pre>
</body>
</html>