<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body><div>On Sat, Apr 2, 2016, at 10:10 AM, neandr wrote:<br></div>
<blockquote type="cite"><div> As you see with the project name, the project will have a strong
relation to Mike Conley's previous project he run in 2013.
Unfortunately it was not finished, but it's great to have Mike on
board again for the University project. Please have a look on <a href="https://github.com/mikeconley/thunderbird-ensemble/wiki">his
GIT Wiki</a> which has a revised approach now and will be an
important basis for the group of students at the School of
Engineering and Computer Science, Victoria University of Wellington. <br></div>
</blockquote><div> </div>
<div>For the persistent storage piece discussed in the wiki's high level overview, it may be worth considering using IndexedDB from the SharedWorker instead of having to coordinate with and maintain main-thread code using mozStorage.<br></div>
<div> </div>
<div>A quick list of advantages:<br></div>
<div>* IndexedDB is not Gecko-specific; if Thunderbird migrated to Electron/whatever, the pieces in the SharedWorker would likely continue to work.<br></div>
<div>* IndexedDB is accessible to all workers as well as all main-thread document contexts. While I think having the SharedWorker be the source-of-truth and performer-of-heavy-lifting is good architecture, this could be handy in some cases.<br></div>
<div>* IndexedDB persists values using the structured clone algorithm and has special handling for Blobs/Files that could be really handy for things like storing pictures or other large pieces of data that are only needed on-demand.<br></div>
<div>** For example, when reading a contact record that contains a picture Blob/File reference, only a reference to the File is loaded. So you don't load the 100K image contents nor do you need to re-serialize them when postMessage()ing the blob to page contexts from the worker. And in fact, you can just use URL.createObjectURL() to hand the Blob directly to an <img> tag (and then subsequently revokeObjectURL once the load event fires.)<br></div>
<div>** This kind of thing would arguably be a nightmare to deal with manually using SQLite/mozStorage. If you read the IndexedDB source code, a large amount of it is all of the Blob handling and reference counting and such to make sure the files don't get orphaned/etc.<br></div>
<div>* IndexedDB now has some string collation smarts so if you're doing some of the searching using the database, it can help you with that.<br></div>
<div>* There's some prior art from the Firefox OS mozContacts implementation that can be used for ideas. See <a href="https://dxr.mozilla.org/mozilla-central/source/dom/contacts/fallback/ContactDB.jsm">https://dxr.mozilla.org/mozilla-central/source/dom/contacts/fallback/ContactDB.jsm</a> for its IndexedDB bits and <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/API/Contacts_API">https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/API/Contacts_API</a> for some contacts. That API is going away and has a variety of lessons learned (like don't try to be everything to everyone and have N indices that may or may not be useful to the apps that want to consume it), so don't depend on the existing implementation or treat it as perfect!<br></div>
<div> </div>
<div>Things that might seem like disadvantages but aren't really:<br></div>
<div>* Gecko's SQLite does build with the Full-Text Search (FTS) extension enabled, but its tokenizing infrastructure is limited and really the main benefit is just that the inverted-index is very cleverly encoded so there's a fair amount of compression. The address book's contents will likely constitute several orders of magnitude less data than mail full-text search, so I don't think this matters that much. The smaller data set means tricks like generating prefixes/substrings of names/etc. so IndexedDB range queries can be used for search.<br></div>
<div>* SQLite does allow for cool SQL joins and such, but 1) clever indexes/specialized object stores can accomplish that for IndexedDB and 2) gloda exists and already understands multiple email addresses per contact and all that, so gloda can probably be used in those cases already or with some minor enhancements to add some attributes to the addressbook indexer, etc. <br></div>
<div> </div>
<div>If people decide to go with using SQLite and mozStorage and all that, it might be worth looking to integrate with gloda. Although gloda's main database is intentionally just an index for a variety of reasons, it's got existing mozStorage ORM (Object-Relational Mapping) bits with cascading queries and could also be used to store canonical data in separate databases/etc.<br></div>
<div> </div>
<div>Andrew<br></div>
</body>
</html>