Error when running hhblits: Cannot allocate memory/Could not read index file...
2
0
Entering edit mode
22 months ago

Hi all,

I tried to run hhblits but it gives errors as: Could not read index file...

The command I run is:

hhblits -i input.fa -d /home/yanhuang/Desktop/hhsuite/database/uniclust30_2018_08_hhsuite/uniclust30_2018_08 -o output.hhr -norealign -n 3 -cov 80 -e 0.001

And the output is:

ffindex.c:452 ffindex_index_parse: mlock: Cannot allocate memory

    11:23:20.049 WARNING: In /build/hhsuite-rhvmkl/hhsuite-3.0~beta2+dfsg/src/hhdatabase.cpp:50: FFindexDatabase:

    11:23:20.049 WARNING: Could not read index file/home/yanhuang/Desktop/hhsuite/database/uniclust30_2018_08_hhsuite/uniclust30_2018_08_cs219.ffindex. Is the file empty or corrupted?

ffindex.c:452 ffindex_index_parse: mlock: Cannot allocate memory

    11:23:21.125 WARNING: In /build/hhsuite-rhvmkl/hhsuite-3.0~beta2+dfsg/src/hhdatabase.cpp:50: FFindexDatabase:

    11:23:21.125 WARNING: Could not read index file/home/yanhuang/Desktop/hhsuite/database/uniclust30_2018_08_hhsuite/uniclust30_2018_08_a3m.ffindex. Is the file empty or corrupted?

ffindex.c:452 ffindex_index_parse: mlock: Cannot allocate memory

    11:23:21.148 WARNING: In /build/hhsuite-rhvmkl/hhsuite-3.0~beta2+dfsg/src/hhdatabase.cpp:50: FFindexDatabase:

    11:23:21.148 WARNING: Could not read index file/home/yanhuang/Desktop/hhsuite/database/uniclust30_2018_08_hhsuite/uniclust30_2018_08_hhm.ffindex. Is the file empty or corrupted?

Segmentation fault (core dumped)

And when I tried uniclust30_2018_08 instead of uniclust30_2018_08_hhsuite, similar problem occurs:

ffindex.c:452 ffindex_index_parse: mlock: Cannot allocate memory
- 12:24:37.459 WARNING: In /build/hhsuite-rhvmkl/hhsuite-3.0~beta2+dfsg/src/hhdatabase.cpp:50: FFindexDatabase:

- 12:24:37.459 WARNING:     Could not read index file/home/yanhuang/Desktop/hhsuite/database/uniclust30_2018_08/uniclust30_2018_08_cs219.ffindex. Is the file empty or corrupted?

- 12:24:37.459 ERROR: In /build/hhsuite-rhvmkl/hhsuite-3.0~beta2+dfsg/src/hhdatabase.cpp:40: FFindexDatabase:

- 12:24:37.459 ERROR:   could not open file '/home/yanhuang/Desktop/hhsuite/database/uniclust30_2018_08/uniclust30_2018_08_a3m.ffindex'

The problem could be reproduced on multiple computers, from my desktop computer to my laptop. But one thing I noticed is that when running the same command on other database, everything will be OK.

For example:

hhblits -i input.fa -d /home/yanhuang/Desktop/hhsuite/database/pfamA_32.0/pfam -o output.hhr -norealign -n 3 -cov 80 -e 0.001

gives me everything I need. uniclust30_2018_08 and uniclust30_2018_08_hhsuite are the only ones that would give this problem.

For your information, I am using hhsuite version 3.0.0 (15-03-2015) on Ubuntu 18.04 and my RAM is 32GB. And I have installed it by git clone. Memory overcommit is enabled.

Really appreciate it for your help, thank you.

software error hhsuite hhblits • 1.4k views
ADD COMMENT
0
Entering edit mode
22 months ago
Mensur Dlakic ★ 14k

The two errors are not the same: you have memory warnings only with the first database you tried, while the second gives you ERROR: could not open file. Beyond the non-existing file I think your problem is literally what the warning says: 32 Gb may not enough to open recent HHblits databases. I know for sure that any of my 16 Gb computers can't be used with newest HHblits any more. Speaking of newest version, it may be a good idea to update to hhsuite that is more recent than the database you are trying to use.

ADD COMMENT
0
Entering edit mode
3 months ago
h.uzzaman ▴ 10

Hi, I'm actually seeing the exact same error. The output just contains a bunch of lines saying "Cannot allocate memory" and "count not read index file...empty or corrupted?" exactly as the OP.

For the memory issue: I'm running hhblits_mpi on a compute cluster using 4 to 10 nodes with 96 GB memory each. For the "could not read index file" issue, I have no idea why the file can't be detected, because I'm using absolute paths, and using "ls" on that address does lead to the file.

Here's the code I'm running:

./hh-suite/bin/hhblits_mpi -i /path/to/pfam_databases/sample_fas -d /path/to/pfam_databases/UniRef30_2020_06 -oa3m sample_a3m_wo_ss -n 2 -cpu 68

The input file has just one amino acid sequence of moderate length.

Would be grateful for any help in this regard!

ADD COMMENT
0
Entering edit mode

You really just should be asking this over at their GitHub repo's issue tracker. Might have even been addressed already.

ADD REPLY
0
Entering edit mode

Thanks so much for your reply! I'll post my issues under the GitHub link (the comments that I see there haven't really addressed the issue yet).

ADD REPLY
0
Entering edit mode

Did you see that updating to a higher hhsuite version solved the problem?

I think this problem happens because I was used older version of hh-suite(3.0). I have been solved my problem by re-install this software with version 3.1

ADD REPLY
1
Entering edit mode

Yes I did- the hh-suite I'm running is version 3.3 (compiled from source), which is why I thought this solution doesn't apply in my case.

I've since run some additional experiments, and it seems the problem (memory allocation + unable to use index files) only happens when I run the MPI version of hhblits. Vanilla hhblits or hhblits_OMP don't show this error (even with nothing else changed). I'll make a post in the github forum after checking out a couple other things. But thanks so much for following up thus far!

ADD REPLY
0
Entering edit mode

Did you set up the MPI properly following these suggestions?

ADD REPLY

Login before adding your answer.

Traffic: 1384 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6