Large operations in sync databases

Ben Bucksch ben.bucksch at beonex.com
Tue Jul 13 09:11:16 UTC 2010


  On 12.07.2010 21:41, Kent James wrote:
> I decided to attempt to get around possible blocking of short sync 
> access by long async queries by dividing my message database into two 
> separate files with independent connections.

I take it sqlite is not multi-threaded? If it was, it should take care 
of the problem. If it isn't, I am not sure two databases solve the 
problem, as sqlite may still block even across databases. Have you tried 
both variants?

It's a real pity when one has to resort to such "optimizations".

(For a hilarious example, which I do not suggest to be comparable, see 
<http://thedailywtf.com/Articles/The-Certified-DBA.aspx>.)

> One file "db" has the metadata represented as a single text field, 
> accessed by an indexed string itemId. This file will be accessed in 
> sync operations, that would typically just be to get the text metadata 
> blob for a single message given its itemId.

Is there a very good reason to not use async? It's fairly easy to use 
with current Mozilla, esp. if you write yourself a small convenience 
wrapper in JS, and we're used to async function calls, given that all 
network requests like XMLHttpRequest are async as well, and we do that 
all the time, nobody would use sync there. Similarly, DB access should 
always be async as well. I found it not so hard to do.


---

In case you're worried about your code readability (like I was) when 
using async, here's a little trick I use to keep the sequential reading 
of statements from top to bottom:

...
doSomething();
var select = ...
db.runQuery(select,
function(results) // success callback
{
   var processed = munge(result);
   continueDoingSomethingHere(processed);
   doOtherThing();
   ...
},
function(e) // error callback
{
   ...
});


That's unusual code formatting, but it has the for me important 
advantage to be able to read from top to bottom and read statements in 
execution order without jumping up and down between independent 
functions, nor check if there's anything else calling this independent 
continuance function.

So, by using closures, you can nest the callback in your function, and 
just read from top to bottom and get execution order.



More information about the tb-planning mailing list