Invitation for technical discussion on next-generation Thunderbird

Mon Apr 24 00:30:11 UTC 2017

On 4/22/17 8:59 PM, Ben Bucksch wrote:
> I don't like compiling/transpiling JS. I like linting and optional 
> compilers.
>
> closure-compiler might be an option, because it's optional and could 
> be applied only on release or as lint step.
>
> Other suggestions?

As far as I'm aware, TypeScript is the only commonly-used thing for 
typing JS. It's probably not too hard to make an optional typing system 
that we can check using a static checker--but trying to invent our own 
JS type system will definitely be one of those things that drives 
contributors away far more than anything else I propose. I worry that 
the options here are "typescript" or "no static typing, ever", and I 
think I would personally lean towards the former. Compilation steps 
aren't all that annoying.

>>  think that establishing tests against a few open source server 
>> implementations with common data setups in Docker containers is the 
>> best way to test these components
>
> Yup, that's a nice idea. But nothing I'd want to run on my machines. 
> I'd want to trigger a "try" build and test server for that. Or just 
> land and see whether it passes (see above).

The ability to run tests on your local machine is something that I 
consider, as a developer, non-negotiable. If you've been reading today's 
IRC backlog, you'll see some painful testing happening where the 
developer in question doesn't have access to the thing being tested--and 
when you have 40-minute 
wait-for-results-oops-I-fat-fingered-a-variable-name cycles, that is 
going to crash developer productivity. Having integration tests largely 
be against targets that I can easily test with on my machine is quite 
superior, IMO.

>
> * There might not be one single solution for all DB needs in TB:NG. It 
> may differ from case to case, so we should evaluate it with specific 
> cases where we need DBs and why and what the requirements are. You 
> probably have some cases in mind, so why don't you just dump your 
> brain somewhere?

I have an old start at <https://old.etherpad-mozilla.org/tb-new-db-api>, 
although that was mostly focused on a better way of representing the 
current API in a more asynchronous manner. I've definitely advocated in 
the past for even stronger changes (most notably, per-account databases 
rather than per-folder databases, which I think opens us up to better 
optimizations in cases like deleting messages), but that's hard to 
retrofit in the current API assumptions (again, assuming that loading a 
database gets you O(instant) access to any message really cripples what 
you can do in the DB design space).

> This is more detail about APIs and implementation that we should 
> discuss on this thread, or even at this time. But your question was 
> valid and important, so I wanted to answer. I hope this removes your 
> concern.

Honestly, no. I'm concerned about performance, and in terms of database 
performance, it is insanely hard to beat Mork (at least for accessing 
via primary key only). Mork is basically a hash table of row ID -> hash 
table of key -> value stores, and all strings are interned (which means 
matching is simple 1-cycle integer comparison, not dozens-cycle string 
comparison). A JSON file would pretty much be like mork, except string 
values aren't going to be interned, and the keys will only be interned 
if there's shape homogeneity (which we'll lose if we allow arbitrary 
keys in the hash table like the database does now)--in other words, 
JSON-backed database is going to show somewhat worse performance 
characteristics than our current mork databases do. And unfortunately, 
we already know that we have performance issues with mork--which means 
the problem isn't in the implementation, it's in the approach.

This means that the API design needs to start from the assumption that 
"a current list of all messages in the folder" is not something that 
should be presumed to exist as an in-memory representation, which means 
filtering even a conceptual list has to let the filter be expressed in 
some manner that lets the query be passed down to a DB layer (i.e., not 
a generic JS function parameter). As you say in other messages, the key 
to performance is not doing things that don't need to be done, and 
getting a list of 1M messages so you can show 20 of them on the screen 
is an excellent example of failing that criterion. It's worth noting 
that this use case is explicitly something that came up as "why writing 
this in HTML/JS is a bad idea" on the Hacker News post about the 
rewrite-in-JS tb-planning post.
>
> Ben
>
>
> _______________________________________________
> tb-planning mailing list
> tb-planning at mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning

-- 
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/tb-planning/attachments/20170423/7d9dfb73/attachment-0001.html>