Question: (Closed) minia assembler - finished with log assembly: 70%
0
gravatar for thackl
4.3 years ago by
thackl2.6k
MIT
thackl2.6k wrote:

Hi,

I just ran a minia assembly on a 3GB plant genome. Everything went smooth so far and I got contigs. However, in the log its says

[assembly] 70 % elapsed: 1042 min 31 sec  estimated remaining: 446 min 47 sec 
minia

in the last line before final stats. Do the 70% mean that something went wrong and things terminated prematurely?

minia • 1.6k views
ADD COMMENTlink modified 4.3 years ago • written 4.3 years ago by thackl2.6k

Is there a separate error log file? Maybe an error was displayed to stdout? This does look like premature termination.

ADD REPLYlink written 4.3 years ago by RamRS20k

No, no errors or something similar - that is what confuses me.

ADD REPLYlink written 4.3 years ago by thackl2.6k

Hello thackl!

We believe that this post does not fit the main topic of this site.

The assembly terminated prematurely due to an NFS exception not connected to the minia assembler.

For this reason we have closed your question. This allows us to keep the site focused on the topics that the community can help with.

If you disagree please tell us why in a reply below, we'll be happy to talk about it.

Cheers!

ADD REPLYlink written 4.3 years ago by thackl2.6k
3
gravatar for edrezen
4.3 years ago by
edrezen720
France
edrezen720 wrote:

Hello,

Indeed it seems a little bit strange. In order to try to understand, could you tell

  1. the version of minia you use
  2. the command line you used
  3. the full stats you got in the command line at the end of the execution
  4. you should have got a tool named "dbginfo" in the minia distribution; could you try "dbginfo -in yourfile.h5" and send the result ?

Actually, it doesn't look like a crash since it seemed you got final stats. The displayed percentage corresponds to the number of branching nodes (in the de Bruijn graph) that have been processed so far and there is no reason why it doesn't go to 100%.

ADD COMMENTlink written 4.3 years ago by edrezen720
1
gravatar for thackl
4.3 years ago by
thackl2.6k
MIT
thackl2.6k wrote:

1. minia-0.1.1 (It was already installed - I just saw, there is v1.0.3 available)

2. minia \
 -in il-01.fq,il-02.fq,il-03.fq,il-04.fq \
 -kmer-size 31 \
 -out minia-01 \
 -out-dir minia-01 \
 -max-memory 100000 \
 -nb-cores 31

3.

    input                                   
        -in                                      : il-01.fq,il-02.fq,il-03.fq,il-04.fq
        -traversal                               : monument
        -starter                                 : best
        -contig-max-len                          : 10000000
        -bfs-max-depth                           : 500
        -bfs-max-breadth                         : 20
        -kmer-size                               : 31
        -abundance                               : 3
        -bank-convert                            : tmp
        -out                                     : minia-01
        -out-dir                                 : minia-01
        -verbose                                 : 1
        -max-memory                              : 100000
        -max-disk                                : 0
        -nb-cores                                : 31
        -bloom                                   : cache
        -debloom                                 : cascading
        -branching-nodes                         : stored
        -mphf                                    : none
    stats                                   
        traversal                                : monument
        start_selector                           : best
        nb_contigs                               : 5706141
        nb_small_contigs_discarded               : 16937817
        nt_assembled                             : 640246631
        max_length                               : 14688
        max_length_left                          : 13719
        max_length_right                         : 11291
    debugging traversal stats               
        couldn't validate consensuses            : 4596
        large bubble breadth                     : 244
        large bubble depth                       : 661
        stopped at marked kmer                   : 4277466
        no kmer extension                        : 73627
        in-branchin large depth                  : 1286943
        in-branching large breadth               : 1693114
        in-branching other                       : 4033741
    time                                     : 66263.691
        assembly                                 : 66263.691
4.
graph                                  
    gatb-core-library                        :    
        version                                  : 1.0.5
        build_date                               : 2014-12-07 04:45:49
        build_system                             : Linux-3.16.0-4-amd64
        build_compiler                           : /usr/bin/cc  (4.9.1)
        build_kmer_size                          : 32 64 96 128
    dsk                                      :    
        config                                   :       
            kmer_size                                : 31
            abundance                                : 3
            available_space                          : 9304175
            bank_size                                : 111738
            sequence_number                          : 727274793
            sequence_volume                          : 111738
            kmers_number                             : 95347730379
            kmers_volume                             : 727445
            max_disk_space                           : 335214
            max_memory                               : 100000
            nb_passes                                : 1
            nb_partitions                            : 155
            nb_bits_per_kmer                         : 64
            nb_cores                                 : 31
            partition_type                           : 0
            nb_cores_per_partition                   : 1
            nb_partitions_in_parallel                : 31
        stats                                    :       
            kmers_nb_distinct                        : 3306584899
            kmers_nb_solid                           : 1150271578
            kmers_nb_weak                            : 2156313321
            kmers_percent_weak                       : 65.2
            fillsolid_time                           : 1021.531         
                1.read                                   : 779.100
                2.sort                                   : 158.989
                3.dump                                   : 83.442
        time                                     : 3347.592      
            fill_partitions                          : 2109.983
            fill_solid_kmers                         : 1237.609
    bloom                                    :    
        stats                                    :       
            kind                                     : cache
            bitsize                                  : 6941164544
            nb_hash                                  : 4
            nbits_per_kmer                           : 6.034370
        time                                     : 93.784      
            build_from_kmers                         : 93.784
    debloom                                  :    
        stats                                    :       
            kind                                     : cascading
            size                                     : 9811137808
            nbits_per_kmer                           : 8.529409
            cfp                                      : 2869973264         
                bloom2                                   : 2279565312
                bloom3                                   : 387835008
                bloom4                                   : 127369872
                set                                      : 75203072
        time                                     : 821.271      
            cascading                                : 127.392
            fill_debloom_file                        : 348.423
            finalize_debloom_file                    : 345.456
    branching                                :    
        stats                                    :       
            nb_branching                             : 90209237
            percentage                               : 7.8
        time                                     :       
            build                                    : 437.596

 

ADD COMMENTlink modified 4.3 years ago • written 4.3 years ago by thackl2.6k
1

Okay, I just took a closer look at the contig file and the last sequence is corrupted as well. So it seems, something caused a crash. I will rerun the data set and see if the problem is persistent.

 

Thanks for your help so far

ADD REPLYlink written 4.3 years ago by thackl2.6k
1

Ok, please let me know about your new run.

I don't see much wrong in the information you got. Getting all the stats at the end of minia means that it didn't crash, so I don't see what happened. The only possibility is that the loop over the branching nodes is stopped at 70% because reading them from the h5 file is wrongly considered as finished (something bad in the h5 file ?)

If you still have the h5 file, you could try the following to check how many branching nodes are actually in the h5 file (the h5dump command should also be in the minia distrib):

h5dump -d /branching/nodes yourfile.h5 | head -30

Then, the value at line starting by "DATASPACE" should be the number of branching nodes, 90209237 in your case (according to the dump of dbginfo)

ADD REPLYlink written 4.3 years ago by edrezen720
1

I think I found the problem. I checked dmesg on the node that minia ran on. There are a couple of  "NFS: state manager:" errors in the log right about the time the run finished. So you are probably right, minia lost connection to the h5 file during processing. Sorry for bugging you with this...

I installed minia-1.0.3 and will rerun the assembly. If I encounter any problems, I will get back to you

ADD REPLYlink written 4.3 years ago by thackl2.6k
1

Okay, the original assembly has been interrupted. This time, it finished at 100%.

Thanks for your help.

ADD REPLYlink written 4.3 years ago by thackl2.6k

Thank you for your feedback.

ADD REPLYlink written 4.3 years ago by edrezen720
Please log in to add an answer.
The thread is closed. No new answers may be added.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1837 users visited in the last hour