Inreasing memory usage in bcftools
0
0
Entering edit mode
3.7 years ago

Hello

I am a beginner in ngs bioinformatic analysis. I try to create a multisampled .vcf file from 250 .bam file using mpileup bcftools. Here the command I use:

bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam

There are two questions I want to ask about this.

  1. I use Linux machine with 32GB of memory, but only 10% of RAM being used by the system. Is there any way to increase the memory usage in the bcftools command line?

  2. Can we show the expected time or progress bar in the bcftools command line? I was tried to insert

    bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam | pv -p -t -e but the time and bar didn't visualized the progress.

I hope you can help me to solve my problems.

Thank you

RAM usage bcftools expected time progress bar • 2.3k views
ADD COMMENT
1
Entering edit mode

but only 10% of RAM being used by the system.

I don't think bcftools mpileup uses a lot of memory

Can we show the expected time or progress bar in th

no, but you can always show what's happening

bcftools mpileup -f ref_genome.fasta bam_file_list*.bam | tee /dev/tty | bgzip > output.vcf.gz

ADD REPLY
0
Entering edit mode

@Pierre Lindenbaum

Thank you for the command-line advice, now I can see what's happening during the process.

ADD REPLY
0
Entering edit mode

but only 10% of RAM being used by the system

Not using a lot of RAM is a good thing. In general, (bio)informatic processes are either constrained by:

  • Computational tasks/CPU (check htop on your machine to see processor usage)
  • Memory usage (as you see, not a lot in this case)
  • input output: reading and writing data from/to disk
ADD REPLY
0
Entering edit mode

Thanks for your comment @WouterDeCoster

Can we accelerate the process by increasing the RAM usage?

ADD REPLY
1
Entering edit mode

The tool will probably be constrained by one of the other parameters, probably CPU usage. You could launch multiple processes in parallel, for example doing variant calling separately per chromosome.

ADD REPLY
0
Entering edit mode

...and how many BAMs are you actually passing to BCFtools? The use of bam_file_list*.bam looks like a risky maneuver to me. You can supply a list of BAMs to BCFTools mpileup with:

-bam-list FILE
ADD REPLY

Login before adding your answer.

Traffic: 1269 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6