Question: MetaVelvet gets stuck at "Starting pebble resolution..." step (individually and probably as a part of MetAMOS)
0
gravatar for Tim
4.9 years ago by
Tim110
United Kingdom
Tim110 wrote:

 

Hello 

I am using Ubuntu 64-bit 14.04 LTS with 32 Gb of RAM and running Velvet v1.2.10 and MetaVelvet v1.2.02 with MAXKMERLENGTH=31. I have the following problem: MetaVelvet gets stuck at the "Pebble resolution" (scaffoliding) step when trying to assemble contigs from a test dataset of 100000 Illumina MiSeq 2x300 PE reads, which were not processed in any way (no trimming, adapter removal etc.). After reaching "Starting pebble resolution..." step meta-velvetg gets stuck for indeterminable amount of time with one core showing 100% load. The last time I tried to test it I waited for more than 12 hours before killing the process manually.

I also tried to use MetaVelvet as one of the assemblers in the MetAMOS pipeline and it froze with the same symptoms at the (probably) same step.

 

​$ meta-velvetg Assem -ins_length 350

[meta-velvetg] Check command line options...
    OK. Your command line options seem to be good.

[meta-velvetg] Load meta-graph ...
[0.000001] Reading read set file Assem/Sequences;
[0.048786] 200000 sequences found
[0.222462] Done
[0.593372] Reading graph file Assem/Graph2
[0.593503] Graph has 148377 nodes and 200000 sequences
[meta-velvetg] Category = 'short1'    Ave. = 350, SD = 35
[meta-velvetg] Category = 'short2'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'long'    Ave. = -1, SD = -1
[meta-velveth] ...done (load meta-graph).

[meta-velvetg] Estimate coverage parameters...
[mate-velvetg] Estimate expected coverage ... yes.    Expected coverage = 1.87805
[mate-velvetg] Estimate expected coverages ... yes.
[1.431262] Writing into stats file Assem/meta-velvetg.Graph2-stats.txt...
[MetaHisto] First valley = 2
[MetaHisto] Largest peak coverage = 2 (frequency count = 6.35635e+06)
[MetaHisto] Noise cutoff coverage = 635635

[meta-velvetg] Warning: Can't find multiple coverage peaks.
[meta-velvetg] Trun on single coverage peak mode. 

[MetaGraph] 1-th coverage peak = 1.87805
[meta-velvetg] Estimate coverage cutoff ... yes.    Coverage cutoff = 0.939024
[meta-velvetg] ...done (estimate coverage parameters).

[meta-velvetg] Remove low & high coverage nodes ...
[meta-velvetg] Min. coverage cutoff for short reads        = 0.939024
[meta-velvetg] Min. coverage cutoff for long reads         = -1
[meta-velvetg] Max. coverage cutoff for short & long reads = -1
[meta-velvetg] Min. contig length                          = -1
[VelvetGraph]  === Remove low coverage nodes ===
[1.642372] Removing contigs with coverage < 0.939024...
[1.645049] Concatenation...
[1.655673] Renumbering nodes
[1.655685] Initial node count 148377
[1.656127] Removed 0 null nodes
[1.656145] Concatenation over!
[1.659085] Concatenation...
[1.669341] Renumbering nodes
[1.669351] Initial node count 148377
[1.669692] Removed 0 null nodes
[1.669698] Concatenation over!
[VelvetGraph]  === Remove high coverage nodes ===
[VelvetGraph]  === Clip tips hardly ===
[1.669714] Clipping short tips off graph, drastic
[1.675634] Concatenation...
[1.686590] Renumbering nodes
[1.686609] Initial node count 148377
[1.686959] Removed 0 null nodes
[1.686967] Concatenation over!
[1.686984] 148377 nodes left
[meta-velvetg] ...done (remove low & high coverage nodes).

[meta-velvetg] Scaffolding based on paired-end information ...
[MetaGraph]  === Scaffolding with single peak mode ===
[VelvetGraph]  === Rock Bank  ===
[1.687007] Read coherency...
[1.690673] Identifying unique nodes
[1.693808] Done, 82558 unique nodes counted
[1.693821] Trimming read tips
[1.701690] Renumbering nodes
[1.701704] Initial node count 148377
[1.702079] Removed 0 null nodes
[1.702087] Confronted to 0 multiple hits and 0 null over 0
[1.702092] Read coherency over!
[VelvetGraph]  === Create read paring array ===
[VelvetGraph]  === Detach dubious reads ===
[VelvetGraph]  === Activate gap markers ===
[VelvetGraph]  === Scaffolding ===
[1.703117] Starting pebble resolution...

 

Any help would be really appreciated.

ADD COMMENTlink modified 4.8 years ago by Anaïs Vittu0 • written 4.9 years ago by Tim110
2
gravatar for rtliu
4.9 years ago by
rtliu2.0k
New Zealand
rtliu2.0k wrote:

1. try Trimmomatic for adapter and quality trimming.

2. recompile Velvet with MAXKMERLENGTH=191 (or even bigger kmer)

3. rerun with k = 127 or 191 (ideal kmer should be 1/3nd to 2/3nd of read length)

Good luck:)

ADD COMMENTlink modified 4.9 years ago • written 4.9 years ago by rtliu2.0k

Thank you for your suggestions, rtilu, but none of them worked, unfortunately. I tried (in different combinations):

  • removing adapters and doing quality trimming of reads (I used BBTools instead of Trimmomatic, in my tests BBTools showed better performance than Trimmomatic);
  • just trimming reads to a specified length (to 100 bp);
  • running velvetg and meta-velvetg with "--scaffolding no" parameter;
  • recompiling Velvet and MetaVelvet with "MAXKMERLENGTH=303";
  • re-running Velvet and MetaVelvet with a whole range of different kmer values;
  • running MetaVelvet with different reference sets of reads.

Every time I ran MetaVelvet, the run ended up in the exactly same way - freezing at the "Starting pebble resolution..." step.

As many people previously used MetaVelvet with success, I started to suspect that something is wrong with my computer and/or Linux installation. Thus I installed Velvet and MetaVelvet under a virtual machine running Ubuntu 12.04.5 LTS and MetaVelvet worked perfectly that time. I also noticed that two different versions of g++ were used to compile Velvet and MetaVelvet: g++ 4.6.3 in Ubuntu 12.04 (the version that worked) and g++ 4.8.2 in Ubuntu 14.04. So I suspect that this behaviour is due to the differences in these two versions of g++ compiler. 

 

TL;DR: MetaVelvet doesn't run properly under Ubuntu 14.04 when compiled with g++ 4.8.2, but works in Ubuntu 12.04.5 (g++ 4.6.3).

ADD REPLYlink written 4.9 years ago by Tim110
1

Omega was recently released as an overlap-graph de novo Assembler for Metagenomics

http://bioinformatics.oxfordjournals.org/content/30/19/2717

or try IDBA-UD, which was pretty good for metagenomic assembly.

http://i.cs.hku.hk/~alse/hkubrg/projects/idba_ud/index.html

 

ADD REPLYlink written 4.9 years ago by rtliu2.0k

Thank you for recommendations, rtliu, these two programs already were my top two alternative choices. I am also pondering about trying SPAdes as well. But I really needed MetaVelvet output for comparison, as it is (probably) the most popular metagenomic assembler nowadays.

ADD REPLYlink written 4.9 years ago by Tim110

Have you used parameter "exp_cov auto"?  As the last resort write to MetaVelvet author. By the way, I have tried SPAdes on one lane HiSeq metagenomic data but it ran of memory on my server.

ADD REPLYlink written 4.9 years ago by rtliu2.0k

Yes, I used this parameter every time I ran the assembler, it didn't help. I am almost sure that this behaviour is related to the version of g++ compiler, so I will let the developers of MetaVelvet know about this issue soon.

As I am working with MiSeq dataset, I am inclined to try SPAdes, read many good things about it.

ADD REPLYlink written 4.8 years ago by Tim110

Did you try by downgrading g++ to 4.6.3?

Thanks!

ADD REPLYlink written 3.8 years ago by armando.torre0
0
gravatar for Anaïs Vittu
4.8 years ago by
European Union
Anaïs Vittu0 wrote:

Hello,

I have the same problem : it blocks at "Starting pebble resolution..." and I use a Mac.

Do someone find a way to fix this ?

 

anais$ meta-velvetg Velvet_out

[meta-velvetg] Check command line options...
    OK. Your command line options seem to be good.

[meta-velvetg] Load meta-graph ...
[0.000000] Reading read set file Velvet_out/Sequences;
[0.575730] 2500000 sequences found
[2.926542] Done
[5.324953] Reading graph file Velvet_out/Graph2
[5.325003] Graph has 116327 nodes and 2500000 sequences
[meta-velvetg] Category = 'short1'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'short2'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'long'    Ave. = -1, SD = -1
[meta-velveth] ...done (load meta-graph).

[meta-velvetg] Estimate coverage parameters...
[mate-velvetg] Estimate expected coverage ... yes.    Expected coverage = 1.64596
[mate-velvetg] Estimate expected coverages ... yes.
[9.092905] Writing into stats file Velvet_out/meta-velvetg.Graph2-stats.txt...
[MetaHisto] First valley = 2
[MetaHisto] Largest peak coverage = 2 (frequency count = 1.85845e+06)
[MetaHisto] Noise cutoff coverage = 185845

[meta-velvetg] Warning: Can't find multiple coverage peaks.
[meta-velvetg] Trun on single coverage peak mode.

[MetaGraph] 1-th coverage peak = 1.64596
[meta-velvetg] Estimate coverage cutoff ... yes.    Coverage cutoff = 0.822981
[meta-velvetg] ...done (estimate coverage parameters).

[meta-velvetg] Remove low & high coverage nodes ...
[meta-velvetg] Min. coverage cutoff for short reads        = 0.822981
[meta-velvetg] Min. coverage cutoff for long reads         = -1
[meta-velvetg] Max. coverage cutoff for short & long reads = -1
[meta-velvetg] Min. contig length                          = -1
[VelvetGraph]  === Remove low coverage nodes ===
[9.373288] Removing contigs with coverage < 0.822981...
[9.375317] Concatenation...
[9.382159] Renumbering nodes
[9.382177] Initial node count 116327
[9.382566] Removed 0 null nodes
[9.382570] Concatenation over!
[9.384559] Concatenation...
[9.391212] Renumbering nodes
[9.391231] Initial node count 116327
[9.391626] Removed 0 null nodes
[9.391630] Concatenation over!
[VelvetGraph]  === Remove high coverage nodes ===
[VelvetGraph]  === Clip tips hardly ===
[9.391658] Clipping short tips off graph, drastic
[9.394651] Concatenation...
[9.401089] Renumbering nodes
[9.401101] Initial node count 116327
[9.401495] Removed 0 null nodes
[9.401500] Concatenation over!
[9.401502] 116327 nodes left
[meta-velvetg] ...done (remove low & high coverage nodes).

[meta-velvetg] Scaffolding based on paired-end information ...
[MetaGraph]  === Scaffolding with single peak mode ===
[VelvetGraph]  === Rock Bank  ===
[9.401533] Read coherency...
[9.402940] Identifying unique nodes
[9.404654] Done, 53448 unique nodes counted
[9.404666] Trimming read tips
[9.407842] Renumbering nodes
[9.407857] Initial node count 116327
[9.408259] Removed 0 null nodes
[9.408267] Confronted to 0 multiple hits and 0 null over 0
[9.408278] Read coherency over!
[VelvetGraph]  === Create read paring array ===
[VelvetGraph]  === Detach dubious reads ===
[VelvetGraph]  === Activate gap markers ===
[VelvetGraph]  === Scaffolding ===
[9.419840] Starting pebble resolution...

 

ADD COMMENTlink modified 4.8 years ago • written 4.8 years ago by Anaïs Vittu0

Anaïs Vittu, could you please post the version of C++ compiler you used for installation of  MetaVelvet?

ADD REPLYlink written 4.8 years ago by Tim110

Hi,

My version of c++ is 4.8.3 (MacPorts gcc48 4.8.3_1).

ADD REPLYlink written 4.8 years ago by Anaïs Vittu0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1984 users visited in the last hour