BLAST Database error: Error pre-fetching sequence data
2
0
Entering edit mode
3.2 years ago
elesbb91 ▴ 60

So, I am trying to use the human_genomic_transcript database. This, for some reason, is not on the NCBI FTP website, however, according to this it should be. So instead, I went to https://www.ncbi.nlm.nih.gov/assembly/GCF_000001405.39 and at the top right area of the page there is "download assembly". So I clicked it. Downloaded the "RNA FASTA", "RNA from genomic FASTA", and "CDS from genomic FASTA".

I then executed:

makeblastdb -in GCF_000001405.38_GRCh38.p12_cds_from_genomic.fna -parse_seqids -out CDS_HUMAN -dbtype nucl

makeblastdb -in GCF_000001405.39_GRCh38.p13_rna.fna -parse_seqids -out HUMAN_RNA -dbtype nucl

makeblastdb -in GCF_000001405.39_GRCh38.p13_rna_from_genomic.fna -parse_seqids -out HUMAN_RNA_GENOMIC -dbtype nucl


These successfully run. No errors given. However, if I look in the directory there is an *.ndb-lock file. I have no idea why it is there, it is not there for the other databases I have built from the fasta, so I am wondering if this is the cause for the error in the title.

Anyone have any ideas?

Thanks!

software error assembly blast ncbi • 6.8k views
0
Entering edit mode

Do you have the error while making a blastn search? maybe the problem is with your query sequences rather than with your database ;)

1
Entering edit mode

Yes, it is when doing the "blastn" search, specifically, dc-megablast task. However, the error clearly states "BLAST Database error" :) so I don't think its my query sequences (besides, they were taken from NCBI and simply pasted to a text file).

BUT BUT BUT BUT!

If I build my database WITHOUT "-parse_seqids" parameter, IT WORKS! I also don't know exactly what the -parse_seqids does... The .fna file has the ">sequence info here" setup, and when doing the blastn search, it shows that sequence info in the results.. so I am not sure why that parameter is required?

0
Entering edit mode

Do you have spaces in the fasta headers descriptors? You possibly do. -parse_seqids is probably dropping text after first space making them non-unique. You can remove spaces in fasta headers (replace them _) and try again to see if the error goes away.

0
Entering edit mode

There are spaces yes. This is default from the NCBI assembly tho.

Here's the output now, when using "makeblastdb" this is the error: BLAST Database creation error: Near line 1, the local id is too long. Its length is 183 but the maximum allowed local id length is 50. Please find and correct all local ids that are too long.

So I am guessing it was stopping after the first space.

Here is the first sequence in the file I am trying to make into a DB:

>lcl|NC_000001.11_cds_NP_001005484.1_1_[gene=OR4F5]_[db_xref=CCDS:CCDS30547.1,GeneID:79501]_[protein=olfactory_receptor_4F5]_[protein_id=NP_001005484.1]_[location=69091..70008]_[gbkey=CDS]
ATGGTGACTGAATTCATTTTTCTGGGTCTCTCTGATTCTCAGGAACTCCAGACCTTCCTATTTATGTTGTTTTTTGTATT
CTATGGAGGAATCGTGTTTGGAAACCTTCTTATTGTCATAACAGTGGTATCTGACTCCCACCTTCACTCTCCCATGTACT
TCCTGCTAGCCAACCTCTCACTCATTGATCTGTCTCTGTCTTCAGTCACAGCCCCCAAGATGATTACTGACTTTTTCAGC
CAGCGCAAAGTCATCTCTTTCAAGGGCTGCCTTGTTCAGATATTTCTCCTTCACTTCTTTGGTGGGAGTGAGATGGTGAT
CCTCATAGCCATGGGCTTTGACAGATATATAGCAATATGCAAGCCCCTACACTACACTACAATTATGTGTGGCAACGCAT
GTGTCGGCATTATGGCTGTCACATGGGGAATTGGCTTTCTCCATTCGGTGAGCCAGTTGGCGTTTGCCGTGCACTTACTC
TTCTGTGGTCCCAATGAGGTCGATAGTTTTTATTGTGACCTTCCTAGGGTAATCAAACTTGCCTGTACAGATACCTACAG
GCTAGATATTATGGTCATTGCTAACAGTGGTGTGCTCACTGTGTGTTCTTTTGTTCTTCTAATCATCTCATACACTATCA
TCCTAATGACCATCCAGCATCGCCCTTTAGATAAGTCGTCCAAAGCTCTGTCCACTTTGACTGCTCACATTACAGTAGTT
CTTTTGTTCTTTGGACCATGTGTCTTTATTTATGCCTGGCCATTCCCCATCAAGTCATTAGATAAATTCCTTGCTGTATT
TTATTCTGTGATCACCCCTCTCTTGAACCCAATTATATACACACTGAGGAACAAAGACATGAAGACGGCAATAAGACAGC
TGAGAAAATGGGATGCACATTCTAGTGTAAAGTTTTAG


I really need all the info so I can't shorten the first line.. Also, I remember making a database out of the human genomic fasta I downloaded without any issues and that had spaces and such?

0
Entering edit mode

I see an *.ndb-lock file in the directories"xmc_cds_database.ndb-lock". Although the command did work in computing platform, it failed in my pc(ubuntu18.04 LTS in win10).

and when i del the option" -parse_seqids" ,it works here.

what's wrong with my ubuntu

0
Entering edit mode
[ws@BIG /mnt/g/project/sipanchong/NCBI_genome/xianmaochong]\$ makeblastdb -dbtype nucl -in /mnt/g/project/sipanchong/NCBI_genome/xianmaochong/xianmaochong_cds.fa -title xmc_cds -out xmc_cds_database -parse_seqids

Building a new DB, current time: 05/18/2020 22:59:03
New DB name:   /mnt/g/project/sipanchong/NCBI_genome/xianmaochong/xmc_cds_database
New DB title:  xmc_cds
Sequence type: Nucleotide
Deleted existing Nucleotide BLAST database named /mnt/g/project/sipanchong/NCBI_genome/xianmaochong/xmc_cds_database
Keep MBits: T
Maximum file size: 1000000000B

**terminate called after throwing an instance of 'lmdb::corrupted_error'
what():  mdb_dbi_open: MDB_CORRUPTED: Located page was wrong type
Aborted (core dumped)**


I see an *.ndb-lock file in the directories"xmc_cds_database.ndb-lock". Although the command did work in a computing platform, it failed in my pc(ubuntu18.04 LTS in win10).

and when I del the option" -parse_seqids", it works here.

what's wrong with my ubuntu

4
Entering edit mode
3.2 years ago
elesbb91 ▴ 60

Okay, @genomax,

So I didn't include -parse_seqids. And when I type the command: blastdbcmd -db RNA_HUMAN -entry NR_118908.1 I get: Error: [blastdbcmd] DB contains no accession info.

Out of curiosity I downloaded v2.9.0+ and and using that makeblastcmd everything works. Including the -parse_seqids. I can search and everything is as expected. I even reinstalled and it still didn't work.

So for now, I am going to use v2.9.0+. I may not in the future but I was right about -parse_seqids being the first line to each sequence identified by ">". This really isn't the best solution, so I am going to contact NCBI to see if they have any info for me.

EDIT WITH SOLUTION So, the real problem is due to: BLASTDB_LMDB_MAP_SIZE=1000000

This environment variable is necessary for the makeblastdb command. BUT, the larger your files the larger that size needs to be. If I set it to 1GB, everything works perfectly fine and runs and its just exactly perfect and what is expected.

Thanks all for the help and maybe this will help someone else

0
Entering edit mode

This has helped someone else, cheers!

1
Entering edit mode
3.2 years ago
GenoMax 127k

BLAST Database creation error: Near line 1, the local id is too long. Its length is 183 but the maximum allowed local id length is 50. Please find and correct all local ids that are too long.

Since the error message is clear you don't have any option but to see if you can stay in that limit. -parse_seqid flag is needed for associating the results with NCBI's taxonomic database. I don't think you need to do that so you could omit that option.

If you are trying align NGS data to transcriptome you could look into aligners like GMAP/GSNAP as an alternative.

0
Entering edit mode

Ahhhhh. I completely read that wrong. I thought you needed -parse_seqid for if the sequences had a local id. But that -parse_seqids is only needed to pair the local id with a supplied taxid file.. I see now!

Thank you thank you. So that fixes my error but doesn't make me understand why the .ndb-lock file would exist once the process is completed. But that may just be an "under the hood" kind of thing.

Thanks genomax!