Data storage for remote content whitelist

Andrew Sutherland asutherland at asutherland.org
Tue Sep 6 23:03:31 UTC 2011


On 09/06/2011 03:44 PM, Joshua Cranmer wrote:
> Of the various storage types, my experiences with them:
> SQLite. As Taras has pointed out, SQLite has an atrocious memory 
> lookup pattern. There also appears to be a tendency to just waste 
> space (right now, empty places files are initialized to a whopping 
> 10MiB, although a single vacuum cleans that up to a few hundred K). 
> Timing data for an SQL backend indicates that, as a {key,key}->value 
> store, it is pretty atrocious, not to mention the high overhead in the 
> API to manage the database.

The 10 MiB is an intentional measure to avoid the SQLite file being 
fragmented on the disk.  If we only grew the file as new pages were 
written, it is likely the file-system would not be able to reliably 
allocate contiguous pages.  The growth in 10 MiB increments is only done 
on systems with at least 500 MiB of free space.


> IndexedDB. Judging from asuth's comments, this actually isn't usable 
> from chrome code. But it layers on top of the backend, so you 
> essentially get to leave it up to other people how to actually 
> implement it.

I think I left out a sentence on there on my reply.  It is usable from 
chrome code, just not in JS modules or anything else lacking a real 
'window' global.  You need to load it into an iframe or use a hidden 
frame.  If the place you load it uses the system principal then there 
may be another trivial bug in core to fix regarding its permission 
checking logic.

Andrew



More information about the tb-planning mailing list