Question: Fastq-Dump Crashes, how to prevent besides continually restarting?
0
gravatar for dec986
12 months ago by
dec986130
United States
dec986130 wrote:

Hello,

I'm downloading public data files from NCBI using (for example)

fastq-dump --split-files -F --bzip2 SRR2637659 > SRR2637659.out 2> SRR2637659.err

however, these commands crash every 6 runs or so, and download very very slowly. So far the only solution is to simply start again when fastq-dump fails.

The problem is that after I leave work I cannot restart this, or when I'm sleeping. Is there any option or clever trick so I don't have to keep manually re-starting this every time fastq-dump fails?

or even better yet, is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set? that would simply downloading greatly as well.

thanks

fastq-dump sra ncbi • 340 views
ADD COMMENTlink modified 12 months ago • written 12 months ago by dec986130
1

What's the error it's failing with? For downloading all at once, look at this answer.

Edit: Got beat to it, his solution is easier.

ADD REPLYlink modified 12 months ago • written 12 months ago by jared.andrews071.6k
1
gravatar for genomax
12 months ago by
genomax59k
United States
genomax59k wrote:

is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set?

Use EBI-ENA to directly download fastq files for your accessions avoiding these issues. As an example your accession # above.

ADD COMMENTlink modified 12 months ago • written 12 months ago by genomax59k

unfortunately, EBI-ENA has different issues. The "Bulk Download Files" button produces an error on Firefox "The address wasn’t understood

Firefox doesn’t know how to open this address, because one of the following protocols (jnlp) isn’t associated with any program or is not allowed in this context.

You might need to install other software to open this address."

I tried this same page on Safari and it tells me to do this on Firefox :/

However, I solved this: I should note that their downloading help page isn't so straightforward https://www.ebi.ac.uk/ena/browse/read-download when you need a * in order for wget to work (Silly me)

thank you @genomax!

ADD REPLYlink written 12 months ago by dec986130

You must have some local restriction on Java/jnlp code or don't have Java installed. When it works bulk download is convenient. For future reference: You can also click on the TEXT link to get a spreadsheet that has the actual FTP links that can be parsed when you have multiple samples in a bioproject.

ADD REPLYlink modified 12 months ago • written 12 months ago by genomax59k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1062 users visited in the last hour