Invitation for technical discussion on next-generation Thunderbird (Semantic Desktop: Pyramid of Users including for Privacy)
pdfernhout at kurtz-fernhout.com
Sun Apr 30 17:01:40 UTC 2017
On 28.04.2017 09:49, Gervase Markham wrote:
> On 25/04/17 04:26, Paul D. Fernhout wrote:
>> So, for me, a good assumption is that people will eventually have
>> billions of message items comprising terabytes of data in their local
>> (shared-with-email) store.
> I don't think that's a good assumption at all, because the "delete"
> button exists. Who has access to their text messages from 10 years ago?
> I think I do, but then I'm a data migration geek. I doubt one person in
> 100 does. People don't want to manage this much data.
It would be helpful to clarify what you mean by "People"? All people?
Almost all people? Most people? Some people? A few people?
For example of why such clarity matters, one of the biggest mistakes
Mozilla made in the Firefox UI redesign was misunderstanding the pyramid
of users of any application. That was a mistake which is part of falling
Firefox marketshare and likely will result in upcoming massive
(negative) changes for Mozilla as search revenue contracts dry up.
Ideally Thunderbird would not repeat that same mistake.
Essentially, in very round numbers, there is a pyramid of users for any
major extensible application:
* <0.1% of an apps user base are the developers
* 1% of the user base is power users who customize the app for
themselves and makes plugins and answer deeper questions and strongly
advocate for it; they also test heavily thorugh their own use, report
bugs, and may make some improvements to the core
* 9% of users are fairly savvy and also answer simpler questions and use
at most a few of the customization features
* 90% of users just use it out of the box because it is popular and well
One of Mozilla's major mistakes with Firefox (in order to chase Chrome's
user base) was to say, essentially, we don't need to support the 1% of
users who want to customize it and heavily use it in new ways. Mozilla
decided to focus on the 99% of users because that is easier and
customization just causes confusion for some users and supporting all
the legacy customizations of the 1% would be hard (and Mobile and IoT
were the new shiny things money went into instead). And guess what --
that 1% of Firefox power users got really annoyed and started leaving
the platform (at least emotionally). And it turns out, there goes all
the volunteers who who had made Mozilla a success in the first place by
evangelizing Firefox and testing it and reporting bugs and creating an
ecosystem around it. For example, I used to install Firefox for family
and friends on desktops -- now I just tell people to buy a Chromebook.
TB:NG is facing similar design decisions right now. Is TB:NG for the
Thunderbird power users who care about maintaining Thunderbird some of
whom have archives of millions of messages? If we decide not to support
power users, then why bother with doing more than reskinning Nylas Mail
To make a point, this is my first time posting to this list from ISP's
RoundCube webmail from my Chromebook instead of Thunderbird (although I
have used that webmail service on-and-off for years from laptops, BCCing
myself like now so Thunderbird POP on my desktop eventually gets a copy,
given Thunderbird's lack of a server component). RoundCube works for
email. It's a feasible solution for email for most people -- even if it
is not everything I want.
=== Deletion takes energy
Deletion is management! If you want to make things easy for users,
either delete everything automatically on a schedule as it comes in or
delete nothing ever. Deciding what to delete selectively is a very hard
problem involving a lot of human effort and anxiety. To what value when
disk is so cheap?
That's probably one other reason you still have all those chat messages
-- it would take more effort and emotion to delete most of them than it
does to just keep them all and search them locally when you want.
And further, as Pat Helland wrote of append-only big data but applies as
well to local data, given systems that delete stuff are more fragile:
"Accountants Don’t Use Erasers"
I guess I'm naturally biased towards preservation rather than deletion.
I'm also a trustee for my historical society trustee -- and even as we
keep up some old buildings, I'm concerned about the loss of our digital
history as well and the fact that almost no one seems to care about it.
Yet, the 1% who do care (often not the financial 1%) can make a big
difference for the community.
Theodore Sturgeon (who wrote the Skills of Xanadu previously mentioned
which inspired Ted Nelson and Hypertext and so on) also coined a law:
"ninety percent of everything is crap".
I won't disagree that, say, we could make the web itself a lot easier to
use (and more accurate) for most users by deleting 90% of it. :-) But
rather than censorship, we use search engines and social processes and
semantic tagging and other tools instead. By not deleting the 90% of the
web most of us don't need, we avoid deleting the 10% (or less) of
content that really matters if we had guessed wrong. TB:NG as a
comprehensive messaging platform should adopt the same strategy.
=== Video is an increasingly bigger part of messaging
On-demand movie apps and video conferencing apps are typically
implemented through processing streams of short messages. Any regular
user can reach millions of messages handled (or stored) fairly quickly
these days. In round numbers, a 1 GB (1,000,000 K) movie or video
conference stream divided by 100K per message for a video frame is
10,000 messages. Watch 100 movies a year or have 100 video conferences
per year for ten years, and that is 10,000,000 messages handled.
Now you may say, but wait, TB:NG does not need to handle streaming video
messages. Well, that's a design choice. But why make that choice without
discussion -- especially given steaming video is an increasing part of
corporate communications and now a new feature of Slack? Why not think
about how text messaging (including annotations, summaries, chat
messages, and of course emails) interacts with video?
=== Privacy value in local stores to keep search local
And of course, why shouldn't people be able to subscribe to thousands of
busy mailing lists for years just in case to search later maybe? Why
should we assume key information will be on the web versus discussions?
My secret weapon as a software developer years ago was using DejaNews to
look up technical information from Usenet. Now I use a search engine and
look at discussions in web forums, blogs, or issue trackers. But why
should I not want to have a local copy of all Bugzilla issues for all
apps I use continuously updates via the equivalent of an RSS feed? Why
should I want to be dependent on someone else's web servers for
searching for issues of interest?
From a privacy standpoint, it makes much more sense to subscribe to lots
of lists and also download their full archives locally and search
locally than it does to give up private (even strategic) information
about what I am thinking about at the moment by using a search engine.
When I was at IBM Research around the turn of the century (among other
things, helping develop a forerunner to Siri called the IBM Personal
Speech Assistant, implementations of XML standards like XSL-FO, and a
digital video system that embedded HTML in videos), I always had to
think twice about whatever I typed into a search query as far as what
IBM Confidential information a search might disclose. Makes me realize a
potential market for a TB:NG handling billions of messages may actually
be big research companies who want to keep searches in-house.
While I can appreciate the limits of small companies in maintaining a
reliable IT infrastructure, I still don't get why many bigger companies
use Slack instead of hosting chat in-house with, say, Mattermost, or why
the smaller companies don't host Mattermost with third parties. I
outline some of the privacy risks of Slack here:
Now that it looks like ISPs can sell user browsing data, thinking about
alternative ways to support privacy by just downloading everything and
searching locally seems worthwhile to me. But that means we need a
personal messaging system that scales.
My family has a personal library of 1000s of books. One of the
surprising values in that library in a digital age is knowing that
corporations and government are not looking over my shoulder when I read
those books. Sadly I'm increasingly allergic to dust from books, so I
use a Kindle more and more -- and I don't like that situation. Why not
just download all of, say, Project Gutenberg and then choose what I read
privately on a great e-reader? And then why not download all discussions
about all those books and then privately follow the threads I am
Also, I have been on mailing lists where email archives went away. Why
not get archives now, before, say, Google decides to paywall or delete
or even alter them?
=== organizing all our messages is worthwhile
Of course, I am also all for writing great software that helps in
prioritizing what we look at, of what we have, if we have billions of
message stored locally. But that is a different issue (and one involving
Library Science and "The Discipline of Organizing" and more).
"This book changed my view of organizing as the dull, tedious task of
putting things in order into thinking of it as the marvelous study of
how people add structure to things ... Glushko is the master of the
discipline of organizing, painting the fascinating story of how
different organizational schemes change our behavior and our thoughts."
So, if organizing millions or even billions of messages a challenge?
Yes. But for the right people it is an interesting and worthwhile
==== Most users don't care, but that does not matter
You'd be right to say 90% to 99% of users don't care about privacy.
Sigh. But as above, it is the 1% of users who do who could make TB:NG a
success by evangelizing it, testing it, fixing it, and so on.
--Paul Fernhout (pdfernhout.net)
"The biggest challenge of the 21st century is the irony of technologies
of abundance in the hands of those still thinking in terms of scarcity."
More information about the tb-planning