Invitation for technical discussion on next-generation Thunderbird (Semantic Desktop)

Paul D. Fernhout pdfernhout at kurtz-fernhout.com
Tue Apr 25 03:26:59 UTC 2017


The following is why I feel TB:NG should be designed to support billions
of messages of a wide variety of types.

My perspective is from wanting to create tools to support a "social
semantic desktop" of which email is a subset of the items. Other types
of items include news messages, RSS feed messages, chat messages,
notification messages, IoT messages, blog posts, pingbacks, saved web
pages, commands constructing complex shared documents, VR world update
messages, code snippets, notes, and more. Email is a special case of a
much larger world of messages.

So, for me, a good assumption is that people will eventually have
billions of message items comprising terabytes of data in their local
(shared-with-email) store. Eventually each user would be part of a
global federation of users that together have billions of times that
number of messages as a federated semantic web.

I understand that such a vision of messaging on that scale is not yet
the goal of most people in this discussion right now. But I'd still urge
people to think a little beyond redoing Thunderbird as-it-is and try to
imagine a personal messaging system that processes, archives, indexes,
and recalls vast numbers of items every day in interaction with a user
and automated systems as part of a larger federation of similar systems
using shared semantic standards.

Such standard data would include (as mentioned by Phillip) simple things
like mailing list headers which are usually ignored as well as much more
complex ways of tagging and annotating data items by various interactive
and automated processes.

Not that anybody probably knows what a "semantic desktop" will look like
eventually, but some pointers are here:
https://en.wikipedia.org/wiki/Semantic_desktop

This states the challenge well:
http://semanticweb.org/wiki/Semantic_Desktop
"The Internet, electronic mail, and the Web have revolutionized the way
we communicate and collaborate - their mass adoption is one of the major
technological success stories of the 20th century. We all are now much
more connected, and in turn face new resulting problems: information
overload caused by insufficient support for information organization and
collaboration. For example, sending a single file to a mailing list
multiplies the cognitive processing effort of filtering and organizing
this file times the number of recipients - leading to more and more of
peoples' time going into information filtering and information
management activities. There is a need for smarter and more fine-grained
computer support for personal and networked information that has to
blend the boundaries between personal and group data, while
simultaneously safeguarding privacy and establishing and deploying trust
among collaborators. ..."

This is the sci-fi story from 1956 about a local-and-global wearable
messaging system that inspired many early technologist including Ted
Nelson of Project Xanadu and Hypertext fame and so gave us email and the
web and yet still leaves room to aspire towards:
"The Skills of Xanadu" by Theodore Sturgeon
https://archive.org/stream/galaxymagazine-1956-07/Galaxy_1956_07#page/n117/mode/2up
https://archive.org/details/pra-BB3830.08

When I applied to Mozilla back in 2011 to work on Thunderbird, moving
Thunderbird in a Social Semantic Desktop direction was what I aspired
to. I did not understand then that Mozilla was on its way to abandoning
Mozilla Messaging and Thunderbird saying such did not have the potential
to "have an industry-wide impact".

Since then, we have unfortunately seen the emergence a proprietary Slack
app which is eating more and more of organizational communications as it
moves into that messaging niche.

There are other related efforts like Matrix.org and Mattermost that are
steps in a Slack-like direction in various ways, and Apache has some
tangentially related stuff too like RocketMQ, and Ward Cunningham has
moved to the "Smallest Federated Wiki". But they are all still not quite
pushing the big picture of a semantic desktop (although such steps
perhaps they could be adapted in that direction).

Granted, it would be fair to say current Thunderbird users have a
pressing email issue right now that needs to be solved expediently --
and doing anything else risks wandering off into a Neverneverland of
failed dreams like, say, Chandler (on which was spent millions).

That's a frequent problem in scoping projects and deciding what amount
of risk is acceptable for what reward. That's one reason "Software is
Hard" as was said about the failed Chandler project:
http://www.gamearchitect.net/Articles/SoftwareIsHard.html
"Scott Rosenberg coins this as Rosenberg's Law:  Software is easy to
make, except when you want it to do something new.  The corollary is,
The only software that's worth making is software that does something new."

We know we can build a new email client because we can point to
literally dozens of examples of working email clients including
Thunderbird itself. The risks there may be about people actually doing
the job well, but the risks are not about the concept or the standards
because we know they deliver value to people already. SO, building a
TB:NG that is essentially the old Thunderbird seems like a safe way to
spend a million dollars (or whatever by the time it is done). But, as
with Rosenberg's law, if we can point to all those existing systems, and
especially most recently Nylas Mail 2.0, it's harder to say yet another
system is worth writing at all.

This sentiment is reflected in several comments in the Hacker News
article R. Kent James pointed to, for example, by "atonse":
https://news.ycombinator.com/item?id=14156251
"Honestly when I was reading this post, I was thinking "Why don't they
just use Nylas? Nylas has already done this" – it just seems like a
perfect case of not-invented-here. I really don't see the point of
Thunderbird reinventing the wheel to essentially build another Nylas"

It seems unlikely we could ever get a larger group of reasonably
risk-averse people currently gathered together over a love of a certain
email client to agree to pursue a specific radically different vision
requiring significant resources like a social semantic desktop. That can
be true even if (big if) after such a project was completed successfully
the same people might reasonably switch to it.

But if it were possible, one might ask what it would take to convince
people to rally around such an idea as the future of the Thunderbird
community? Somehow I feel at minimum it would take a working prototype
as a necessary but not necessarily sufficient precursor.

That was what I aspired to with the short-lived Twirlip2 experiment
towards a Thunderbird Server. It was short-lived not because it was a
bad idea, just short-lived as other work required my attention and now I
have some better ideas in the meantime. I'm taking another pass at a
prototype here (up to seven, learning something every time, this one is
just starting, with an eye to sharing code snippets at first):
https://github.com/pdfernhout/Twirlip7

This is also one reason I scrapped the Twirlip2 approach of thinking
plugins could be in the same repo instead of, say, using npm packages
for each plugin, as suggested in an essay by the creator of Octopress:
http://octopress.org/2015/01/15/octopress-3.0-is-coming/
"If I'm being harsh, I'll tell you that as it is now, Octopress is
basically some guy's Jekyll blog you can fork and modify. The first, and
most obvious flaw, is that Octopress is distributed through Git. I want
to punch through a wall when I think about users wrestling with merge
conflicts from updating their sites. It’s absurd.
  Octopress is released as a single product, but it's a collection of
plugins and configurations which are hard to disentangle. if you want to
change or remove anything you're leaving the "golden path" and updates
will be painful, if not impossible — short of copy and paste. Even I
can't remove things from Octopress. If I want to stop maintaining a
plugin it will also disappear for anyone who updates. While some have
suggested using Git tags as a kind of steam-punk versioning, that simply
will not solve the real problem. This isn't how software products should
be distributed. Git is for collaborators; not users. ... This release is
a full rewrite. Octopress as it is now will be replaced by a selection
of gems, each semantically versioned with is own documentation and
tests. ..."

A future Thunderbird messaging platform should have a similar
architecture, where you could ideally "npm install" support for a new
message type or new indexer and so on.

Anyway, my two cents.

--Paul Fernhout (pdfernhout.net)
"The biggest challenge of the 21st century is the irony of technologies
of abundance in the hands of those still thinking in terms of scarcity."

On 2017-04-24 1:48 PM, Phillip Hallam-Baker wrote:
> While there are many folk with very large inboxes, the number of people
> with really large numbers of personal mail is rather smaller.​ So
> separating the mail out and storing it separately is useful. While I
> don't want every mail on every mailing list I subscribe to on every
> device, I do want all my personal mail.
> 
> One caveat is that while most of my messages by number are from mailing
> lists, my personal mail is much larger by data volume because of
> attachments.


More information about the tb-planning mailing list