Question: Inreasing memory usage in bcftools
0
gravatar for rizky_dsatrio
7 months ago by
rizky_dsatrio0 wrote:

Hello

I am a beginner in ngs bioinformatic analysis. I try to create a multisampled .vcf file from 250 .bam file using mpileup bcftools. Here the command I use:

bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam

There are two questions I want to ask about this.

  1. I use Linux machine with 32GB of memory, but only 10% of RAM being used by the system. Is there any way to increase the memory usage in the bcftools command line?

  2. Can we show the expected time or progress bar in the bcftools command line? I was tried to insert

    bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam | pv -p -t -e but the time and bar didn't visualized the progress.

I hope you can help me to solve my problems.

Thank you

ADD COMMENTlink written 7 months ago by rizky_dsatrio0
1

but only 10% of RAM being used by the system.

I don't think bcftools mpileup uses a lot of memory

Can we show the expected time or progress bar in th

no, but you can always show what's happening

bcftools mpileup -f ref_genome.fasta bam_file_list*.bam | tee /dev/tty | bgzip > output.vcf.gz

ADD REPLYlink modified 7 months ago • written 7 months ago by Pierre Lindenbaum119k

@Pierre Lindenbaum

Thank you for the command-line advice, now I can see what's happening during the process.

ADD REPLYlink written 7 months ago by rizky_dsatrio0

but only 10% of RAM being used by the system

Not using a lot of RAM is a good thing. In general, (bio)informatic processes are either constrained by:

  • Computational tasks/CPU (check htop on your machine to see processor usage)
  • Memory usage (as you see, not a lot in this case)
  • input output: reading and writing data from/to disk
ADD REPLYlink written 7 months ago by WouterDeCoster38k

Thanks for your comment @WouterDeCoster

Can we accelerate the process by increasing the RAM usage?

ADD REPLYlink written 7 months ago by rizky_dsatrio0
1

The tool will probably be constrained by one of the other parameters, probably CPU usage. You could launch multiple processes in parallel, for example doing variant calling separately per chromosome.

ADD REPLYlink written 7 months ago by WouterDeCoster38k

...and how many BAMs are you actually passing to BCFtools? The use of bam_file_list*.bam looks like a risky maneuver to me. You can supply a list of BAMs to BCFTools mpileup with:

-bam-list FILE
ADD REPLYlink modified 7 months ago • written 7 months ago by Kevin Blighe41k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 818 users visited in the last hour