Question: How to download a 100 fastq files at once using SraToolkit fastq-dump?
0
gravatar for flight029000
5 months ago by
flight0290000 wrote:

How to download a 100 fastq files at once using SraToolkit fastq-dump? Is there any software with GUI or shell codes published somewhere?

fastq-dump sratoolkit • 228 views
ADD COMMENTlink written 5 months ago by flight0290000
2

use GNU parallel

Gnu Parallel - Parallelize Serial Command Line Programs Without Changing Them

ADD REPLYlink written 5 months ago by shenwei3563.3k

No, that is not recommended for this type of job! As fastq-dump is mainly a downloader, the limiting factor is mostly network bandwidth and IO. Unless you have a much larger bandwidth on your side than NCBI's load balancer is willing to assign to your ip for each connection, it is most likely to be slower with so many connections. It depends on how their load balancing is implemented but you might even be throttled or blacklisted. I would try max. 4-6 parallel downloads and monitor the network traffic.

ADD REPLYlink modified 6 weeks ago • written 6 weeks ago by Michael Dondrup43k
1

Did you try using a for loop already?

ADD REPLYlink written 5 months ago by b.nota3.5k

no, post an example, pls

ADD REPLYlink written 5 months ago by flight0290000

What did you try already?

ADD REPLYlink written 5 months ago by b.nota3.5k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1500 users visited in the last hour