Entering edit mode
7.8 years ago
stevenlang123 ▴ 200
So I've been trying to
bgzip around 100 VCF files in parallel, but although the jobs are submitted and files get created there's definitely something wrong.
So far I've been trying:
$ for file in *.vcf > do > bsub /foo/bar/bgzip $file >> $file.gz
What is the correct way to do this?
Thanks in advance!
When I use
ls *.vcf | xargs -P 10 bgzipit only compresses the first file in the folder. Using
-n1(=use at most 1 argument per command line) instead of
-P 10worked for me:
You can use both (
xargs -n1 -P0) to get a computer to use max_procs and dispatch one file (line) per process.
Is there any way to do this using an LSF manager to split the jobs up instead ? The problem I'm having is that bgzip requires to redirect the file output so
does not work
Never mind, working great Pierre! Thanks!