Question: Getting BLAST to give output whilst running
1
gravatar for CC
3.5 years ago by
CC30
Melbourne, Australia
CC30 wrote:

I run large BLAST jobs that can sometimes take days or weeks to complete. With other jobs I find I can see the output of the job as it is running, which lets me know it is working and how its progress is doing, however with BLAST it doesn't give any output until it is finished, which means I never really know it's working until it's done. I was wondering if there is any way to get BLAST to flush output as it is running, or some other way for it to show it is working? I am using Version 2.3.0.

blast • 1.2k views
ADD COMMENTlink modified 2.2 years ago by lakhujanivijay4.5k • written 3.5 years ago by CC30

Are you using a job scheduler? Different schedulers handle intermediate output differently. e.g. LSF may buffer the output in a temp location while the job is running. You should be able to look at that output while the job is in progress (e.g. in LSF you would bpeek {job_id}).
If you are running blast standalone it should be writing output as it goes.

ADD REPLYlink written 3.5 years ago by genomax73k

Yes, I am using a job scheduler as I am using a cluster at work. I've been told it is managed by a combination of Torque and Moab... is there a way to look at intermediate output with those schedulers?

ADD REPLYlink written 3.5 years ago by CC30

I am not familiar with torque but this may help.

ADD REPLYlink modified 3.5 years ago • written 3.5 years ago by genomax73k

Thanks, this makes it clear.

ADD REPLYlink written 3.5 years ago by CC30
1
gravatar for lakhujanivijay
2.2 years ago by
lakhujanivijay4.5k
India
lakhujanivijay4.5k wrote:

I don't know if this helps but what I usually do is that I enable desktop notification on my PC; something like this:

A desktop notification on the top right corner

This notification pops up; say every hour and tells me how many of the query sequences have been blasted so far (I already know how many were there in total). So, that provides me an idea of the progress without disturbing me from my current work. Here is a short bash script to achieve the same:

while true; do 
count=`ls -1 *.xml | wc -l`
   if [ $count != 0 ]
   then 
     total_blastx_done=`grep "Iteration_query-def" *.xml -cH`
     notify-send "Blastx compltd: $total_blastx_done"
     sleep 900
   fi 
done

Notice the following:

  1. my blastx output format is xml format i.e. -outfmt 5
  2. I grep the tag "Iteration_query-def" from the xml file. This could be anything, but should be unique for each query sequence.
  3. Script looks for all xml files in the current folder where blast was fired and notify status of all of them.
  4. sleep command takes no. of seconds as input i.e. how frequently you want yourself to be notified. In my case (script above, it is set to 900i.e every 15 minutes). You can set it accordingly.
  5. You can run the script in the background like mentioned below:

$ nohup sh count_blastx.sh &

ADD COMMENTlink written 2.2 years ago by lakhujanivijay4.5k
0
gravatar for Manu Prestat
2.2 years ago by
Manu Prestat3.9k
Marseille, France
Manu Prestat3.9k wrote:

Some answers here: Show Progress (Bar?) For Blast+ Query

ADD COMMENTlink written 2.2 years ago by Manu Prestat3.9k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1507 users visited in the last hour