Question: R And Sqlite Db Locked Error
0
gravatar for polarise
5.1 years ago by
polarise370
Galway, Ireland
polarise370 wrote:

I've become bald pulling out my hair because of this problem. Time and time again I need to run an R package that needs to use RSQLite but fails because of NFS locking. Well, that's the setup on our cluster. Curiously, I don't see other people complaining about this problem but I know that a lot of bioinformatics is done on Sun Grid Engine installations, which I assume employ NFS in one way or another.

Any tips on how to alleviate this pain? Are my assumptions right (about SGE and NFS being inseparable)?

Thanks!

PK

R • 2.5k views
ADD COMMENTlink modified 5.1 years ago by Michael Dondrup45k • written 5.1 years ago by polarise370

This is definitely off topic, but I thank that on serverfault.com you will receive a good answer if you give some more details: e.g. http://serverfault.com/questions/71871/best-system-to-serve-sqlite-databases-to-webservers-over-a-network

ADD REPLYlink modified 5.1 years ago • written 5.1 years ago by Michael Dondrup45k

my experience: sqlite3 doesn't work with SGE/NFS. I tried to log the steps of my NGS workflow with sqlite3, I had to switch to a java standalone sql engine (derby)

ADD REPLYlink written 5.1 years ago by Pierre Lindenbaum114k

Perhaps if I stated that I was running a Bioconductor package it might draw it in... ;-)

ADD REPLYlink written 5.1 years ago by polarise370

If you could also say which package, that may make it bioinformatics

ADD REPLYlink written 5.1 years ago by Michael Dondrup45k

Which R/Bioconductor package are you using and when do you get this message ? (I mean are you trying to parallelize calls to a function that uses RSQLite? )

ADD REPLYlink written 5.1 years ago by Sudeep1.6k
1
gravatar for Sean Davis
5.1 years ago by
Sean Davis25k
National Institutes of Health, Bethesda, MD
Sean Davis25k wrote:

This is a SQLite FAQ.

http://www.sqlite.org/faq.html#q5

ADD COMMENTlink written 5.1 years ago by Sean Davis25k
0
gravatar for Michael Dondrup
5.1 years ago by
Bergen, Norway
Michael Dondrup45k wrote:

Honestly, what I would do in this situation is probably to migrate the package to a server based DBMS by patching the code, e.g. Migrate to RPostgreSQL. The SQL should be mostly compatible and run updates in a transaction.

And did I say already that it would really help to know which package you use? Giving sufficient level of details often helps solving problems, just saying...

ADD COMMENTlink modified 5.1 years ago • written 5.1 years ago by Michael Dondrup45k

I'm using the following package: http://bioconductor.org/packages/release/data/annotation/html/hugene10stprobeset.db.html

The current solution is to install the package on the master node (if accessible) since that will have its own disk hence no NFS lock.

ADD REPLYlink written 5.0 years ago by polarise370
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1643 users visited in the last hour