Windows Cmd Not Responding With Python Blastall 2.2.17

Question

[Niek De Klein]

I'm using the standalone BLAST 2.2.17 with python. However, I'm having a problem with the cmd. When running the script the cmd stops responding at the 15th blast record. It doesn't matter if I change the proteins, it's always the 15th.

from Bio.Blast import NCBIStandalone
from Bio.Blast import NCBIXML


my_blast_db = r"C:\Niek\Test2.2.17\Worm\c_elegans.protein.WS200.fasta"
my_blast_file = r"C:\Niek\Test2.2.17\Worm\worm-HD.fasta"
my_blast_exe = r"C:\Niek\blast-2.2.17\bin\blastall.exe"

result_handle, error_handle = NCBIStandalone.blastall(my_blast_exe, "blastp",
my_blast_db, my_blast_file, matrix="BLOSUM62")



blast_records = NCBIXML.parse(result_handle)

y = 0
#see if target is in TF list

for blast_record in blast_records:
    if blast_record:
        y+=1
        print y

It worked normally like this earlier today and with other databases.

Has anyone else had this problem/know how to fix it?
Thanks,
Niek

Edit: I used blast 2.2.17 in command line directly (using same commands as used above) to create an xml file and read that in as result_handle. That works.

@Michael's comment:

from Bio.Blast.Applications import NcbiblastpCommandline

my_blast_db = r"C:\Niek\Test2.2.17\Worm\c_elegans.protein.WS200.fasta"
my_blast_file = r"C:\Niek\Test2.2.17\Worm\worm-HD.fasta"


blastp_cline = NcbiblastpCommandline(cmd='blastp', query=my_blast_file, db=my_blast_db, evalue=0.01,
                                                                        outfmt=5, out="wormAll-HD.xml")

print blastp_cline
print blastp_cline()

print blastp_cline gives:

blastp -query C:\Niek\Test2.2.17\Worm\worm-HD.fasta -db C:\Niek\Test2.2.17\Worm\c_elegans.protein.WS200.fasta -out wormAll-HD.xml -evalue 0.01 -outfmt 5

When I copy that in the blast-2.2.24+ command line tool it works, but in the script it gives :

File "C:/Niek/Test2.2.24/Worm/test.py", line 11, in <module>
print blastp_cline()
File "C:\Python27\lib\site-packages\Bio\Application\__init__.py", line 472, in __call__
shell=(sys.platform!="win32"))
File "C:\Python27\lib\subprocess.py", line 672, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 882, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

[Lee Katz]

Please consider upgrading to Linux.

[Michael Schubert]

Please consider updating to a current version of BLAST (and maybe BioPython) and see if the problem persists, thus making it easier for us to help you.

[Niek De Klein]

It doesn't happen with blast 2.2.24+

[Niek De Klein]

with the blast 2.2.24+ I have the problem that I get "WindowsError: [Error 2] The system cannot find the file specified", but when I paste the "print cline" result that I used in the command line it does work.

[Michael Schubert]

Can you update your code to the one you are using with 2.2.24+ and paste the print cline here?

Answer 1

[Istvan Albert]

I recommend that you change your code such that it writes the results to a file ,finishes the run closes this file, then you parse out this file.

The truth to this matter is that due to the fact that the reading/writing to the standard output and error is buffered, it is very much possible (although it is rare, but when it happens is maddening) for one stream say the standard output to be waiting for input all the while the program is attempting to write to the other stream (standard error) but it cannot. It is the classical deadlock problem. The subprocess module makes efforts to avoid this scenario, but there are situations where it still happens - especially on Windows a platform that is less covered by open source enthusiasts.

This behavior is independent of programming language, it is just a net effect of running and capturing the output of one tool inside the other (Python). Changing it to a structure that first writes the file then reads it gives you more control and insight into what is actually happening.

Answer 2

[Michael Schubert]

I'm still not sure about the error but you can try the following things (I'll update the list if these don't work):

1. Specify the full path to your blastp executable

2. Using subprocess.Popen instead of executing cline() directly

import subprocess, sys

child = subprocess.Popen(str(blastp_cline), shell=(sys.platform!="win32"))

Answer 3

[Peter]

Your first example used the obsolete deprecated legacy blastall command.

The second example uses BLAST+ (the current version of NCBI BLAST), but has a problem:

from Bio.Blast.Applications import NcbiblastpCommandline

my_blast_db = r"C:\Niek\Test2.2.17\Worm\c_elegans.protein.WS200.fasta"
my_blast_file = r"C:\Niek\Test2.2.17\Worm\worm-HD.fasta"

blastp_cline = NcbiblastpCommandline(cmd='blastp', query=my_blast_file, db=my_blast_db, evalue=0.01,
                                                                        outfmt=5, out="wormAll-HD.xml")

print blastp_cline
print blastp_cline()

You've set the cmd=... option, but blastp is the default anyway. If the BLAST+ binaries are not on the $PATH environment variable (which is probably the case on Windows unless you set this up yourself), you'll get the observed error:

WindowsError: [Error 2] The system cannot find the file specified

The fix is to tell Windows where to look - something like this:

from Bio.Blast.Applications import NcbiblastpCommandline

my_blast_db = r"C:\Niek\Test2.2.17\Worm\c_elegans.protein.WS200.fasta"
my_blast_file = r"C:\Niek\Test2.2.17\Worm\worm-HD.fasta"
my_blast_exe = r"C:\Niek\blast-2.2.29+\bin\blastp.exe"

blastp_cline = NcbiblastpCommandline(cmd=my_blast_exe, query=my_blast_file, db=my_blast_db, evalue=0.01,
                                                                        outfmt=5, out="wormAll-HD.xml")

print blastp_cline
print blastp_cline()

That should work... assuming you adjust the my_blast_exe to match wherever you've put BLAST+ on your machine.