Question: MetaVelvet caught in an infinite loop?
0
gravatar for slvt16
3.7 years ago by
slvt160
United States
slvt160 wrote:

Hello,

I've been trying to run MetaVelvet assembly of an environmental DNA sample (short, paired end, fastq, illumina reads) on a supercomputer cluster, and have been unable to generate output. I am using Velvet 1.2.10 and MetaVelvet -1.1.01. The program will run to completion (in a fraction of a second) on very small files (containing 20,000 sequences each) for both single and paired end reads, and produce a functional meta-velvet contigs file. Vevleth and Velvetg both run successfully on files containing 40,000 sequences each. However, when running MetaVelvetg on these results, the program will run until reaching the wall timer.

I am using the following sequence of commands:

  • mpirun -np $PBS_NP ~/bin/velvet-master/velveth $fol 31 -fastq -shortPaired $sequence1 $sequence2
  • mpirun -np $PBS_NP ~/bin/velvet-master/velvetg $fol -read_trkg yes -scaffolding no
  • mpirun -np $PBS_NP ~/MetaVelvet-1.1.01/meta-velvetg $fol -scaffolding no


Where $sequence1 and $sequence2 point to their respective ends of each sequence, and $fol points to the output directory. I have used a different output directory with each run. I typically run the process with 6 nodes, each with 8 2.66 GHz processors and access to some allocation of 2.62 TB shared memory. I have tried up to 24 of those nodes, and MetaVelvet has never run to completion, so it seems like it's not a matter of processing power.

When I've killed the process, the error file contained the following:

MPI: could not run executable (case #3)
MPI: No details available, no log files found
/opt/torque/4.2.9/spool/mom_priv/jobs/77068.hokieone.SC: line 34: 369997 Killed 
mpirun -np $PBS_NP /home/slvt16/bin/velvet-master/velveth $fol 31 -fastq $sequence1
MPI: could not run executable (case #3)
MPI: No details available, no log files found
/opt/torque/4.2.9/spool/mom_priv/jobs/77068.hokieone.SC: line 35: 374736 Killed 
mpirun -np $PBS_NP /home/slvt16/bin/velvet-master/velvetg $fol -read_trkg yes -scaffolding no

The output log shows that the program is hanging on the ------Scafolding------ step, even when run with the '-scaffolding no' flag. 

I would like to run MetaVelvet on paired end files of ~400,000 reads each, eventually. I'm running out of ideas for troubleshooting, so any help would be appreciated!

 

EDIT: Also on: http://seqanswers.com/forums/showthread.php?t=61120

ADD COMMENTlink modified 3.7 years ago by RamRS20k • written 3.7 years ago by slvt160

Hello slvt16!

It appears that your post has been cross-posted to another site: http://seqanswers.com/forums/showthread.php?t=61120

This is typically not recommended as it runs the risk of annoying people in both communities.

ADD REPLYlink written 3.7 years ago by RamRS20k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1374 users visited in the last hour