Large operations in sync databases
ben.bucksch at beonex.com
Tue Jul 13 09:11:16 UTC 2010
On 12.07.2010 21:41, Kent James wrote:
> I decided to attempt to get around possible blocking of short sync
> access by long async queries by dividing my message database into two
> separate files with independent connections.
I take it sqlite is not multi-threaded? If it was, it should take care
of the problem. If it isn't, I am not sure two databases solve the
problem, as sqlite may still block even across databases. Have you tried
It's a real pity when one has to resort to such "optimizations".
(For a hilarious example, which I do not suggest to be comparable, see
> One file "db" has the metadata represented as a single text field,
> accessed by an indexed string itemId. This file will be accessed in
> sync operations, that would typically just be to get the text metadata
> blob for a single message given its itemId.
Is there a very good reason to not use async? It's fairly easy to use
with current Mozilla, esp. if you write yourself a small convenience
wrapper in JS, and we're used to async function calls, given that all
network requests like XMLHttpRequest are async as well, and we do that
all the time, nobody would use sync there. Similarly, DB access should
always be async as well. I found it not so hard to do.
In case you're worried about your code readability (like I was) when
using async, here's a little trick I use to keep the sequential reading
of statements from top to bottom:
var select = ...
function(results) // success callback
var processed = munge(result);
function(e) // error callback
That's unusual code formatting, but it has the for me important
advantage to be able to read from top to bottom and read statements in
execution order without jumping up and down between independent
functions, nor check if there's anything else calling this independent
So, by using closures, you can nest the callback in your function, and
just read from top to bottom and get execution order.
More information about the tb-planning