post TB 3.1 mailnews backend plans

Ben Bucksch ben.bucksch at beonex.com
Sat Jul 17 23:20:15 UTC 2010


  On 18.07.2010 00:44, David Bienvenu wrote:
> I think modern file systems are a lot better about this. I don't know 
> if anything in the maildir standard handles breaking up large 
> directories. If not, there's a trade-off between following from the 
> standard (and any pre-existing tools/code that knows how to deal with 
> maildir) vs. supporting users with huge numbers of messages on older 
> systems, and I'd lean towards the former. Since the store is 
> pluggable, users with that kind of setup may need to use a different 
> store, or occasionally archive messages. I suspect that's the case today.

FWIW, we currently do have a problem with big INBOXen with lots of big 
attachments, and the mbox size exceeds 2/4 GB, IIRC on FAT.

However, it's not just old, stupid filesystems like FAT. Also modern 
filesystems can get awfully slow, up to unusable, when hit with many 
thousand small files. I experienced that with Cyrus (which also stores 
each mail in a separate file) running on XFS, when my spam folder 
accumulated hundreds of thousands mails. It would take an hour (!) or 
more to delete the folder, and some folders couldn't be deleted at all 
anymore, FS got broken. XFS is generally a very good filesystem, but 
definitely has a weakness when it comes to many small files.

Hopefully, a new implementation of our storage wouldn't have such 
limitations.

If storage implementations are pluggable, I guess there'd be a way to 
select the storage type, e.g. in about:config. If so, you could have a 
subtype, e.g. "maildir-plusplus" and "maildir-tb". The latter would 
optionally have several "cur" dirs, or a similar workaround for this 
problem.



More information about the tb-planning mailing list