How Long Should It Take To Make A Blast Database?
8
5
Entering edit mode
13.6 years ago
Andrea_Bio ★ 2.8k

Hi

I'm trying to install a local copy of polyphen and one of the stages is to make a blastdb from uniref100.

I ran this command last night and its still going nearly 12 hours later. I don't know if this is normal:

==============================================

makeblastdb -in uniref100-formatted.fasta -dbtype prot -out uniref -title 'UniRef100 2010_07, 15-Jun-2010' -parse_seqids

Building a new DB, current time: 11/03/2010 23:51:19 New DB name: uniref New DB title: UniRef100 2010_07, 15-Jun-2010 Sequence type: Protein Keep Linkouts: T Keep MBits: T Maximum file size: 1073741824B

============================================

The program just seems to be sitting there. The uniref file is 233GB so I don't expect it to be quick but I wasn't expecting 12 hours. I am running linux on a virtual machine on an external hard drive with only 1 gig of RAM so its not on the fastest machine in the world either.

Thanks

makeblastdb blast error • 11k views
ADD COMMENT
2
Entering edit mode

Are you running a local installation of PolyPhen-2 ? I had similar problems while running the makeblastdb during PolyPhen-2 installation. Usually it wont take that much time for NCBI-nr/Uniprot.

ADD REPLY
2
Entering edit mode

I tried to install PolyPhen 2, but had the same issue with the Uniref steps, after 2 days or so, the makeblastdb exited on its own with a segmentation fault error message. You may change the BLAST+ version as suggested by Daniel. Please let us know if you were able to install it completely. From my experience, other steps won't take much time.

ADD REPLY
1
Entering edit mode

I'm at 3 days on the uniref database stage!

ADD REPLY
0
Entering edit mode

I'm in the process of making a polyphen2 local installation. I've been told one of the steps takes 4 days??? Is this preparing the pdb database?

ADD REPLY
0
Entering edit mode

What should the program produce and how big should it be? I can see in my directory i have a file called uniref100.00.pin. I've never done this before. I might try and get the output from someone else who might have already done it.

ADD REPLY
0
Entering edit mode

Does anyone have the output of this task downloadable? I've reverted to 2.23 and I still find that it's running for a long time. I'm trying this on my local macbook pro

ADD REPLY
5
Entering edit mode
13.6 years ago

This might be a version issue as well. I've found that several of the BLAST utilities (makeblastdb, blastx in particular) run slower in the latest release (2.2.24+). I haven't taken the time to try and track down what exactly in the data causes the slowdown, but if one of the earlier versions (2.2.23+ or 2.2.22) goes quicker, that's something we need to report to the developers.

ADD COMMENT
2
Entering edit mode

Confirmed. I just tried indexing a 41MB data set using makeblastdb 2.2.24+ and it wasn't a quarter done by 5 minutes. I killed the job, ran it with makeblastdb 2.2.23+ and it finished in about 30 seconds.

ADD REPLY
0
Entering edit mode

Are any of these blast programs optimised to make use of the graphics card cpu (using cuda libraries i think?). That would speed things up a bit

ADD REPLY
0
Entering edit mode

I'll have to admit that my knowledge of hardware and hardware acceleration is quite limited. My guess is no, but I can't be sure.

ADD REPLY
0
Entering edit mode

I might downgrade to 2.2.23+. Do you know what files the program produces?i only have a file called uniref100.00.pin.

ADD REPLY
0
Entering edit mode

For a recent task, I indexed a file called AdhA.A2.ffn and makeblastdb 2.2.23+ created the following files: AdhA.A2.ffn.nhr, AdhA.A2.ffn.nin, AdhA.A2.ffn.nog, AdhA.A2.ffn.nsd, AdhA.A2.ffn.nsi, AdhA.A2.ffn.nsq. When you're working with protein data sets, the extensions begin with "p" instead of "n". To be honest, I've never seen documentation on what each of the index files contains, and my curiosity has never been enough to research the issue. I spent a bit of time looking this morning to no avail. I typically rely on the output from makeblastdb to know whether the file was indexed correctly.

ADD REPLY
4
Entering edit mode
13.6 years ago
Paulo Nuin ★ 3.7k

Depending on your processor speed, 12 hours or more doesn't sound abnormal, specially with you 1Gb memory cap. Using a machine with 64Gb of memory, but a slow clock, it took me some hours to create a local copy of nr and nt.

ADD COMMENT
1
Entering edit mode

Agreed; at first I thought that this did sound slow, but it's the 1GB RAM that's killing you. Write to an external HDD could be slow too.

ADD REPLY
4
Entering edit mode
13.5 years ago
Bamyasi ▴ 150

Do not use version 2.2.24+ use 2.2.23+ instead! Version 24+ is buggy and in some cases may run 1,000 slower than previous releases. Had this reported to NCBI BLAST developers 3 months ago, got a confirmation but still fixed binaries nowhere to be found on NCBI FTP.

ADD COMMENT
1
Entering edit mode
13.6 years ago

Here i have the same problem, I'm also trying to create the uniref100 BLAST database. But it already has a runtime of more than 2 weeks. Using blast 2.2.24+, running on a intel core 2 duo 2.8 ghz with 8gb of ram, ram used 16% and cpu used 100%. I contacted uniprot for a downloadable version of the formated uniref100 database but they don't provide this.

ADD COMMENT
0
Entering edit mode

2 weeks??? Mine has been running for 3 days now on the slowest machine in the world. It sounds like it might be best to go back to blast 2.2.23???

ADD REPLY
1
Entering edit mode
13.0 years ago
Yannick Wurm ★ 2.5k

You're using the -parse_seqids option. That slows things by several orders of magnitude.

ADD COMMENT
0
Entering edit mode
13.5 years ago
Magnus B • 0

I am trying to make the same database for polyphen, and makeblastdb has run for over 1 week. The files are still growing, but I do thing there is something wrong with the latest version of the blast program.

ADD COMMENT
0
Entering edit mode
13.4 years ago
Martijn • 0

Same here. Running for a month and not even half of it processed. Killing it now and will move back to 2.2.23. Thx bamyasi.

ADD COMMENT
0
Entering edit mode
13.0 years ago
Julio514 • 0

Had the same problem. Swtiched back to 2.23 and worked fine!

ADD COMMENT

Login before adding your answer.

Traffic: 1699 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6