Help with samtools

Question

[Chris]

Hi all, would you please have a suggestion for the error below? Thank you so much!

samtools sort -@ 16 -m 32G -o sorted_sample2.bam Library_2Aligned.out.bam
samtools sort: couldn’t allocate memory for bam_mem

I run the command on a server.

Answer 1

[ATpoint]

The memory here in sort is calculated per thread so essentially this is 16 threads * 32GB RAM so 512GB requested RAM -- not going to happen and also not necessary for sorting. Do something like -@ 8 -m 1G, that's by far enough and the improvement in speed if you go higher than that is so minimal that it does not merit the additional resources.

[Chris]

Thanks ATpoint! The first bam file is 66Gb so the first command without adding thread and memory has run for hours:

samtools sort -o sorted_sample2.bam Library_2Aligned.out.bam

[Chris]

I need to use the restroom so my computer was inactive for a few minutes and it lost connection to the server so all work after hours running is gone 🥲. Maybe I need to use batch submit.

[ATpoint]

Not sure who you are talking to. Of course jobs to a server should be submitted and independent from an active ssh connection. Sounds like you should talk to the admin of this server aiming for an introduction to some basics. I am not saying to taunt you, but it's really basics you have to know. Also be sure that you are using the job nodes in case that exists rather than the headnode...

[Chris]

I used batch submit and it took not so long. Thank you!

[rpolicastro]

If you're allowed to use the login node on your server to run tasks you can use tmux to keep a command running regardless of your connection.

[Chris]

My job on the server finished. Thank you!