Question: Problem Running Blast Jobs.
2
gravatar for Anjan
8.4 years ago by
Anjan810
United States
Anjan810 wrote:

I started to run blast, locally on my machine, on 4 files with 1323, 210, 501, 166 fasta sequences each.

For all jobs except the first one blast returned an error:

blastall(7004) malloc: *** mmap(size=2097152) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

The first job is currently still running as I'm writing this.

The exact blastall command used is as follows:

blastall -p blastn -d /Users/anjan/blastdb/nt -i <input.filename.fasta>  -o RPMBBP10.fasta.blast.out -m 8 -b 10000.

The version of Blast is 2.2.17. I'm running this on a Mac server with 2 Quad-core Intel processors with 16GB RAM. Anyone has any ideas on what the source of the problem is?

blast error memory • 4.2k views
ADD COMMENTlink modified 8.4 years ago by Michael Dondrup46k • written 8.4 years ago by Anjan810
3
gravatar for Drio
8.4 years ago by
Drio910
United States
Drio910 wrote:

Is it possible you compiled or downloaded a 32bit version of the blast binary? If compiling from source, try to add "-arch x86_64" in your compiler flags. You should be able to add that in your configure file or directly in your Makefile if there is no configure.

ADD COMMENTlink written 8.4 years ago by Drio910

This is almost certainly the explanation. The nt database is large (several GB) so BLAST will fail on 32-bit systems or when using 32-bit BLAST by mistake. See these troubleshooting notes - http://defindit.com/readme_files/blast_failed_to_allocate.html - and mail list thread with a similar problem - http://lists.apple.com/archives/fortran-dev/2008/Jun/msg00002.html.

ADD REPLYlink modified 1 day ago by RamRS24k • written 8.4 years ago by Neilfws48k
1
gravatar for Michael Dondrup
8.4 years ago by
Bergen, Norway
Michael Dondrup46k wrote:

You are running a pretty outdated version of the Blast binaries! Try the latest version: Blast+ v. 2.2.25 (yours is 2.2.17) from ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ or the latest legacy blast (only if you must, I don't see a reason to use this): ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/LATEST/

ADD COMMENTlink modified 1 day ago by RamRS24k • written 8.4 years ago by Michael Dondrup46k
0
gravatar for Chris Evelo
8.4 years ago by
Chris Evelo10.0k
Maastricht, The Netherlands
Chris Evelo10.0k wrote:

You seem to run out of memory (I guess Pierre realized that and added the "memory" tag for that reason). You could try to ask for less alignments. -b 250 is the default if I remember correctly. Why would you want 10.000 alignments anyway?

ADD COMMENTlink modified 8.4 years ago • written 8.4 years ago by Chris Evelo10.0k

Chris, the sequences I am blasting are used for diagnostics. Typically there are hundreds of these deposited by clinical labs around the world. Hence the (somewhat arbitrary) 10000 limit on the number of hits to return.

ADD REPLYlink written 8.4 years ago by Anjan810

OK makes sense. But you could still try a run without the -b parameter to see whether that causes the problem (along with a number of other things that were suggested)

ADD REPLYlink written 8.4 years ago by Chris Evelo10.0k

yes good point. Thanks.

ADD REPLYlink written 8.4 years ago by Anjan810
0
gravatar for Joachim
8.4 years ago by
Joachim2.8k
San Francisco, California
Joachim2.8k wrote:

The error code tells you that the requested memory was not available (http://lists.apple.com/archives/Darwin-dev/2008/May/msg00028.html).

Try running the jobs separately and use 'top' to monitor the memory footprint of each blast job. I am not sure which permissions you have on the server and whether you got access to other installed memory monitoring tools, so you could use modified versions of Code 2 -- 4 as given in http://joachimbaran.wordpress.com/2011/04/03/flystockdb-preprocessing-improvements/ as a poor man's memory-footprinting solution.

Once you have an idea on your memory needs, you can start dividing your current blast-jobs into more but smaller jobs.

ADD COMMENTlink modified 1 day ago by RamRS24k • written 8.4 years ago by Joachim2.8k
0
gravatar for Pierre Lindenbaum
8.4 years ago by
France/Nantes/Institut du Thorax - INSERM UMR1087
Pierre Lindenbaum122k wrote:

Yes, as Chris said it could be a memory problem. But as the origin of the problem was a call to mmap and the error code=12 is a CORE_FILE_OPEN .

#define CORE_FILE_OPEN    12  /* Error opening file */

it could also be an I/O problem.

ADD COMMENTlink written 8.4 years ago by Pierre Lindenbaum122k

problem does not occur when command is run with a file with 7 sequences. that suggests that it may not be an IO issue....

ADD REPLYlink written 8.4 years ago by Anjan810

I think that is not the error code you are looking for.. handwave The errno 12 above is related to mmap and not the one in ncbierr.h.

ADD REPLYlink written 8.4 years ago by Joachim2.8k

ok Joachim. I was not sure about it. But anyway, an error for mmap could be related to I/O.

ADD REPLYlink written 8.4 years ago by Pierre Lindenbaum122k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1066 users visited in the last hour