Revisiting the War on RDF, especially folder lookup

Joshua Cranmer pidgeot18 at
Mon Jan 21 22:11:04 UTC 2013

It looks like The PGO libxul Crisis, Episode 3: The 4GB Limit is now 
playing on a mozilla-central branch near you. While no one has yet 
proposed it as an option, I doubt it will be long before bring up the 
possibility of killing RDF altogether in mozilla-central. If it is 
brought up, we could be left with a short amount of time to scramble for 
alternatives. SeaMonkey is in a much worse state than Thunderbird is, 
but both of us use RDF in one place so pervasively that removing it is 
difficult. That place is the folder lookup model.

There is a single bug we could fix that would eliminate most of our 
(apparent) use of RDF: bug 441437, the folder lookup service. All 
folders are associated with a unique URI which can be used to get a 
folder by calling nsIRDFService::GetResource. That function is extremely 
magical: the RDF service maintains a weak map of all RDF resources 
(including folders), and will lazily create them if it's not in its map. 
This creation is magical, but it is how resources are expected to be 
created. The problem is you can't tell it to not create a folder when 
you don't want to, and there are times when calls are used by 
Thunderbird code in expectation of a lazy creation.

The folder lookup service, as originally proposed, eagerly built the map 
of all folders by recursively iterating all folders in the tree. This 
turns out to be a problem since iterating folders in IMAP can result in 
actual folder creation, which caused a suite of bugs that forced the 
original implementation to be backed out shortly after it first landed. 
An alternative proposed implementation amounts to a lazy map creation, 
but this starts to run into some issues due to ill-defined semantics, as 
I'll note below.

To properly fix the bug, I think we need to sit down and discuss a topic 
that has been surprisingly evasive in our current documentation: how our 
backend server/folder infrastructure works. With respect to the folder 
lookup service, the important questions that need consensus answer are 
as follows:

1. Who creates folder objects? What API calls are necessary to do it?
2. What process causes the creation of folder objects?
3. What semantics to folder URIs are necessary to support the above 

The answers as currently applies to the codebase are as follows:
1. The RDF service does it. Folder creation and initialization happens 
via specific contracts and nsIRDFResource::Init.
2. This is created on first use of a folder. Even if the folder doesn't 
3. There is a very inherent assumption that folders are '/' delimited, 
and the root folder's URI is identical to the incoming server's URI. In 
addition, some components are expected (in nsMsgDBFolder::AddSubfolder) 
to be normalized to specific case-sensitive variants. And don't get me 
started on which methods expect which URIs to be percent-encoded.

I think the most tenable option for a folder-lookup service that 
replaces the current mess is a lazy one, which looks like this:

getFolderForURI(folderURI, forceCreation) {
   if (folderURI in map)
     return map[folderURI];
   else {
     folder = createFolderObject(folderURI, forceCreation);
     if (folder) map[folderURI] = folder;
     return folder;

(Note that "forceCreation" here has the semantics "if the folder doesn't 
physically exist, then make it happen" with the expectation that it will 
default to false for most people. There are one or two places where we 
actively desire those semantics.) Assuming no one has any objections to 
this implementation strategy, the only hard part is deciding what 
"createFolderObject" looks like. One way that reuses current APIs is 
effectively a "compute parent folder and do parent.addSubfolder(...)" 
call on it, but that loses the forceCreation semantics. In such a 
scenario, it appears that the best course of action is to create a new 
API on nsIMsgFolder or nsIMsgIncomingServer that creates the folder, 
which would retain both parameters.

Thus, the API proposal which I think makes the most sense is the following:
interface nsIFolderLookupService {
   nsIMsgFolder getFolderForURI(in AUTF8String folderURI, [optional] in 
bool forceCreation);
interface nsIMsgIncomingServer {
   nsIMsgFolder makeFolder(in AUTF8String folderURI, in bool 

Using this as a baseline, the answers to the questions become this:
1. Folder objects are created by the server, via the API above.
2. Folder objects are created on first-use by the folder lookup service.
3. The authority component of the URI is the URI of the server object; 
the filepath attributes are left to be interpreted by server 
implementations as they choose.


Joshua Cranmer
News submodule owner
DXR coauthor

More information about the tb-planning mailing list