Inreasing memory usage in bcftools

Question

[rizky_dsatrio]

Hello

I am a beginner in ngs bioinformatic analysis. I try to create a multisampled .vcf file from 250 .bam file using mpileup bcftools. Here the command I use:

bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam

There are two questions I want to ask about this.

1. I use Linux machine with 32GB of memory, but only 10% of RAM being used by the system. Is there any way to increase the memory usage in the bcftools command line?

2. Can we show the expected time or progress bar in the bcftools command line? I was tried to insert

   bcftools mpileup -O z -o output.vcf.gz -f ref_genome.fasta bam_file_list*.bam | pv -p -t -e but the time and bar didn't visualized the progress.

I hope you can help me to solve my problems.

Thank you

[Pierre Lindenbaum]

but only 10% of RAM being used by the system.

I don't think bcftools mpileup uses a lot of memory

Can we show the expected time or progress bar in th

no, but you can always show what's happening

bcftools mpileup -f ref_genome.fasta bam_file_list*.bam | tee /dev/tty | bgzip > output.vcf.gz

[rizky_dsatrio]

@Pierre Lindenbaum

Thank you for the command-line advice, now I can see what's happening during the process.

[WouterDeCoster]

but only 10% of RAM being used by the system

Not using a lot of RAM is a good thing. In general, (bio)informatic processes are either constrained by:

• Computational tasks/CPU (check htop on your machine to see processor usage)
• Memory usage (as you see, not a lot in this case)
• input output: reading and writing data from/to disk

[rizky_dsatrio]

Thanks for your comment @WouterDeCoster

Can we accelerate the process by increasing the RAM usage?

[WouterDeCoster]

The tool will probably be constrained by one of the other parameters, probably CPU usage. You could launch multiple processes in parallel, for example doing variant calling separately per chromosome.

[Kevin Blighe]

...and how many BAMs are you actually passing to BCFtools? The use of bam_file_list*.bam looks like a risky maneuver to me. You can supply a list of BAMs to BCFTools mpileup with:

-bam-list FILE