Summary of the situation with the composition process — thoughts wanted

Jonathan Protzenko jonathan.protzenko at gmail.com
Tue Jun 21 18:35:49 UTC 2011


Hi folks,

:bwinton, :jb and I were discussing issues related to the composition 
process, and I thought I might as well post my summary of the situation 
here, so that other people can chime in. :ehsan should give us some 
insight on the situation, and I'm confident other people will have 
interesting things to say.

Below is a (lengthy) summary of the situation, and represents my own 
point of view. If I appreciated wrongly the situation, please do correct 
me asap.

---

The compose window in Thunderbird relies on three broadly defined 
components.

 1. The <editor> component from Gecko; it handles the editable area,
    i.e. where you type your message, the caret, what happens when you
    hit enter, the DOM tree, etc. The code lives in
    comm-central/mozilla/editor/. :ehsan and :kaze are working on it if
    I'm not mistaken.
 2. The editor UI: all the small buttons to insert an image, set text in
    bold, italics, etc. The code lives in comm-central/editor/ui. It's
    horrible code, that hasn't changed for the past 10 years, and unlike
    wine, it doesn't get any better with age. AFAIK, no one's working on
    it, and we definitely need help with it. The number of steps
    required to merely insert an image is complete nonsense, and the
    process is *not intuitive*.
 3. The code for setting up a compose session and sending the message.
    It's all c++, and I'm thinking about
    comm-central/mailnews/compose/src/, most specifically
    nsMsgCompose.cpp and nsMsgSend.cpp.
      * The code first initializes the composition window, does a lot of
        magic, sets up all the composition fields, both visible and
        hidden (recipients, subject, headers, quoted & reformatted text,
        signature, MDN, etc.). It talks to the nsIEditor that the
        <editor> implements to setup html / plaintext editing, the
        encoding, etc.
      * Then, nsMsgSend.cpp kicks in, walks the DOM tree, figures out
        which images should be attached, determines whether html +
        plaintext or just plaintext should be sent, changes the src
        attributes live in the <editor> instance so that <img
        src="blah.jpg"> becomes <img src="cid:whatever"> and then
        serializes it all according to the right encoding, wraps it,
        sends it.

A few months ago, I worked on an experiment to see how much of these we 
could replace easily with JS parts. The goals are, roughly, as follows.

  * Given that we don't have that many resources to devote to the
    composition UI (2.), this would allow us to cheaply get an updated,
    more intuitive UI.
  * Make the composition code more accessible to developers. Hacking
    into that C++ code is insanely hard, the entry cost is high, and
    it's scary. Writing it in JS would make it more concise, lighter,
    and more hackable. We could also drop large chunks of code that make
    no sense today : the C++ code goes great lengths to figure out the
    best encoding to sent the outgoing message with. In compose in a
    tab, I settled for UTF8 always, and saved myself a lot of trouble.
  * Allow experimenting with new designs, such as compose in a tab.

 1. This part doesn't change at all with my experiment.
 2. The Thunderbird UI is replaced by a CKEditor instance.
 3. This is where I come in.
      * I rewrote this part in JS, and I've implemented most required
        actions. This is either code that determines the recipients
        depending on the composition mode, streams the draft to insert
        its body into the edition area, re-uses the attachments from the
        draft, or the forwarded message, etc; or code that performs less
        pleasant tasks, such as rewrapping the text, quoting and
        rewrapping, or convert html to plaintext (do you realize that
        the component in Thunderbird that does html -> plaintext
        conversion for quoting is not even scriptable?).
      * This part heavily relies on an <editor> being available, so I
        had to fake myself into a nsIEditor and a nsIEditorMailSupport.
        This roughly works, but I had to resort to the most vicious
        hacks to get this done (more details in an appendix).

There are many problems, though.

  * CKEditor tries to be cross-platform and hence overrides many builtin
    <editor> features, making them slower and more error-prone. For
    instance, CKEditor will have its own handling of the <Enter> key,
    and will break the DOM tree on its own, move the caret... CKEditor
    has its own spellchecking also. CKEditor is very much heavyweight,
    takes seconds to load, and doesn't necessarily fit well as the UI
    for a mail editor (issues with <blockquote>s).
  * The Herculean undertaking that this represents. There are zillions
    of options and of possible behaviors; more than a sane man would get
    crazy trying to implement them all. I even regularly discover some
    new options myself: the thing with multiple identities for one
    single account, all preferences regarding composition (top-posting
    vs bottom-posting, signatures above/below the quote, signature / no
    signature, quote / no quote, font, color, html, plaintext, html +
    plaintext, utf8, no utf8), s/mime, attachment reminder, MDN, DSN,
    autosave, different composition modes (edit draft, reply, reply all,
    new, etc. I think there are 14 of them), initial attachments, were
    we called through mapi, command-line, drag&drop of attachments from:
    the filesystem, another email, an URL... I believe some of these
    options should go away, even if some users are going to crucify us
    for this. I don't think we have the manpower to undertake a rewrite
    of the composition process and still afford to keep that variety of
    customizations.
  * There are hidden invariants all over the place. Specific design
    patterns that oblige you to re-use a specific object, and modify it
    in place. Very specific calling conventions. Hidden state that
    require you to call just the magic function so that it will fail
    after 10 lines because you're not implementing the right interfaces,
    but will still set the global variable to the right value. More
    generally, the expectation that the only place we will ever compose
    messages from is the composition window.

This pretty much sums up the situation, and I hope this gives a clear 
overview. I've appended some gory details below for those of you who are 
interested. I'd love to have your opinion on this, correct me if you 
think I'm exaggerating, etc. These are statements, but they reflect my 
own feeling of the story, and I'm sure people have different point of 
views to oppose. I'd love to hear about them!

jonathan

Appendix 1 : the thing with global variables

The workflow for sending a message goes like this.
- nsMsgComposeService creates a new nsMsgCompose instance, and creates a 
new composition window (all from C++),
- nsMsgCompose pokes into the editor from the new window, initializes it 
with various parameters, sets its mode to html or plaintext, etc, and 
sets a whole bunch of global variables,
- nsMsgCompose assembles all the parts, queries the editor, and passes 
the control flow to nsMsgSend
- nsMsgSend still pokes into the nsEditor (more specifically, 
nsEditorMailSupport).

This all implies the existence of a composition window. I can't assemble 
the parts myself and talk directly to nsMsgSend because half the 
interfaces there are [noscript]. So what I had to do is use the SendMsg 
function from nsMsgCompose that assembles the parts and moves on to 
nsMsgSend.cpp. However, there are two global variables that have to be 
set if you want to send anything besides plaintext. These two global 
variables are only set when initializing the classic compose window. So 
I had to fake myself into a classic compose window, and implement just 
enough interfaces so that the two variables are initialized correctly, 
and then fail early. nsMsgCompose then *thinks* it's dealing with a 
regular compose session, and goes on with the right editor + html 
settings. This is wicked. (The two variables control whether there's 
html or plaintext (m_composeHTML) and if there's an editor that should 
be poked (m_editor)).

I managed to hack around all these limitations, and somehow build a 
clean API on top of that; it took me months to improve it over and over; 
I consider this an achievement in itself. I'm even able to send 
attachments through JS! This still amazes me. Most of the code went into 
the quick reply feature of Thunderbird Conversations, and this is where 
I keep polishing the code over and over. However, the question of the 
editor UI still remains open.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/tb-planning/attachments/20110621/fc101149/attachment.html>


More information about the tb-planning mailing list