Fastq-Dump Crashes, how to prevent besides continually restarting?
1
0
Entering edit mode
4.0 years ago
dec986 ▴ 310

Hello,

I'm downloading public data files from NCBI using (for example)

fastq-dump --split-files -F --bzip2 SRR2637659 > SRR2637659.out 2> SRR2637659.err

however, these commands crash every 6 runs or so, and download very very slowly. So far the only solution is to simply start again when fastq-dump fails.

The problem is that after I leave work I cannot restart this, or when I'm sleeping. Is there any option or clever trick so I don't have to keep manually re-starting this every time fastq-dump fails?

or even better yet, is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set? that would simply downloading greatly as well.

thanks

SRA fastq-dump NCBI • 1.1k views
ADD COMMENT
1
Entering edit mode

What's the error it's failing with? For downloading all at once, look at this answer.

Edit: Got beat to it, his solution is easier.

ADD REPLY
1
Entering edit mode
4.0 years ago
GenoMax 110k

is there an option that for a given accession number, e.g. GSE88888, I can download all fastq associated with this data set?

Use EBI-ENA to directly download fastq files for your accessions avoiding these issues. As an example your accession # above.

ADD COMMENT
0
Entering edit mode

unfortunately, EBI-ENA has different issues. The "Bulk Download Files" button produces an error on Firefox "The address wasn’t understood

Firefox doesn’t know how to open this address, because one of the following protocols (jnlp) isn’t associated with any program or is not allowed in this context.

You might need to install other software to open this address."

I tried this same page on Safari and it tells me to do this on Firefox :/

However, I solved this: I should note that their downloading help page isn't so straightforward https://www.ebi.ac.uk/ena/browse/read-download when you need a * in order for wget to work (Silly me)

thank you @genomax!

ADD REPLY
0
Entering edit mode

You must have some local restriction on Java/jnlp code or don't have Java installed. When it works bulk download is convenient. For future reference: You can also click on the TEXT link to get a spreadsheet that has the actual FTP links that can be parsed when you have multiple samples in a bioproject.

ADD REPLY

Login before adding your answer.

Traffic: 1115 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6