Inreasing memory usage in bcftools
0
0
Entering edit mode
3.7 years ago

Hello

I am a beginner in ngs bioinformatic analysis. I try to create a multisampled .vcf file from 250 .bam file using mpileup bcftools. Here the command I use:

bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam


1. I use Linux machine with 32GB of memory, but only 10% of RAM being used by the system. Is there any way to increase the memory usage in the bcftools command line?

2. Can we show the expected time or progress bar in the bcftools command line? I was tried to insert

bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam | pv -p -t -e but the time and bar didn't visualized the progress.

I hope you can help me to solve my problems.

Thank you

RAM usage bcftools expected time progress bar • 2.3k views
1
Entering edit mode

but only 10% of RAM being used by the system.

I don't think bcftools mpileup uses a lot of memory

Can we show the expected time or progress bar in th

no, but you can always show what's happening

bcftools mpileup -f ref_genome.fasta bam_file_list*.bam | tee /dev/tty | bgzip > output.vcf.gz

0
Entering edit mode

@Pierre Lindenbaum

Thank you for the command-line advice, now I can see what's happening during the process.

0
Entering edit mode

but only 10% of RAM being used by the system

Not using a lot of RAM is a good thing. In general, (bio)informatic processes are either constrained by:

• Computational tasks/CPU (check htop on your machine to see processor usage)
• Memory usage (as you see, not a lot in this case)
• input output: reading and writing data from/to disk
0
Entering edit mode

Can we accelerate the process by increasing the RAM usage?

1
Entering edit mode

The tool will probably be constrained by one of the other parameters, probably CPU usage. You could launch multiple processes in parallel, for example doing variant calling separately per chromosome.

0
Entering edit mode

...and how many BAMs are you actually passing to BCFtools? The use of bam_file_list*.bam looks like a risky maneuver to me. You can supply a list of BAMs to BCFTools mpileup with:

-bam-list FILE