R And Sqlite Db Locked Error
2
0
Entering edit mode
9.3 years ago
polarise ▴ 380

I've become bald pulling out my hair because of this problem. Time and time again I need to run an R package that needs to use RSQLite but fails because of NFS locking. Well, that's the setup on our cluster. Curiously, I don't see other people complaining about this problem but I know that a lot of bioinformatics is done on Sun Grid Engine installations, which I assume employ NFS in one way or another.

Any tips on how to alleviate this pain? Are my assumptions right (about SGE and NFS being inseparable)?

Thanks!

PK

r • 4.1k views
ADD COMMENT
0
Entering edit mode

This is definitely off topic, but I thank that on serverfault.com you will receive a good answer if you give some more details: e.g. http://serverfault.com/questions/71871/best-system-to-serve-sqlite-databases-to-webservers-over-a-network

ADD REPLY
0
Entering edit mode

my experience: sqlite3 doesn't work with SGE/NFS. I tried to log the steps of my NGS workflow with sqlite3, I had to switch to a java standalone sql engine (derby)

ADD REPLY
0
Entering edit mode

Perhaps if I stated that I was running a Bioconductor package it might draw it in... ;-)

ADD REPLY
0
Entering edit mode

If you could also say which package, that may make it bioinformatics

ADD REPLY
0
Entering edit mode

Which R/Bioconductor package are you using and when do you get this message ? (I mean are you trying to parallelize calls to a function that uses RSQLite? )

ADD REPLY
1
Entering edit mode
9.3 years ago

This is a SQLite FAQ.

http://www.sqlite.org/faq.html#q5

ADD COMMENT
0
Entering edit mode
9.3 years ago

Honestly, what I would do in this situation is probably to migrate the package to a server based DBMS by patching the code, e.g. Migrate to RPostgreSQL. The SQL should be mostly compatible and run updates in a transaction.

And did I say already that it would really help to know which package you use? Giving sufficient level of details often helps solving problems, just saying...

ADD COMMENT
0
Entering edit mode

I'm using the following package: http://bioconductor.org/packages/release/data/annotation/html/hugene10stprobeset.db.html

The current solution is to install the package on the master node (if accessible) since that will have its own disk hence no NFS lock.

ADD REPLY

Login before adding your answer.

Traffic: 746 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6