Question: Fastq-Dump Crashes, how to prevent besides continually restarting?
0
gravatar for dec986
5 days ago by
dec98630
United States
dec98630 wrote:

Hello,

I'm downloading public data files from NCBI using (for example)

fastq-dump --split-files -F --bzip2 SRR2637659 > SRR2637659.out 2> SRR2637659.err

however, these commands crash every 6 runs or so, and download very very slowly. So far the only solution is to simply start again when fastq-dump fails.

The problem is that after I leave work I cannot restart this, or when I'm sleeping. Is there any option or clever trick so I don't have to keep manually re-starting this every time fastq-dump fails?

or even better yet, is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set? that would simply downloading greatly as well.

thanks

fastq-dump sra ncbi • 89 views
ADD COMMENTlink modified 5 days ago • written 5 days ago by dec98630
1

What's the error it's failing with? For downloading all at once, look at this answer.

Edit: Got beat to it, his solution is easier.

ADD REPLYlink modified 5 days ago • written 5 days ago by jared.andrews07480
1
gravatar for genomax
5 days ago by
genomax39k
United States
genomax39k wrote:

is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set?

Use EBI-ENA to directly download fastq files for your accessions avoiding these issues. As an example your accession # above.

ADD COMMENTlink modified 5 days ago • written 5 days ago by genomax39k

unfortunately, EBI-ENA has different issues. The "Bulk Download Files" button produces an error on Firefox "The address wasn’t understood

Firefox doesn’t know how to open this address, because one of the following protocols (jnlp) isn’t associated with any program or is not allowed in this context.

You might need to install other software to open this address."

I tried this same page on Safari and it tells me to do this on Firefox :/

However, I solved this: I should note that their downloading help page isn't so straightforward https://www.ebi.ac.uk/ena/browse/read-download when you need a * in order for wget to work (Silly me)

thank you @genomax!

ADD REPLYlink written 5 days ago by dec98630

You must have some local restriction on Java/jnlp code or don't have Java installed. When it works bulk download is convenient. For future reference: You can also click on the TEXT link to get a spreadsheet that has the actual FTP links that can be parsed when you have multiple samples in a bioproject.

ADD REPLYlink modified 5 days ago • written 5 days ago by genomax39k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1431 users visited in the last hour