Quick download of FASTQ
2
0
Entering edit mode
6 weeks ago
Thanh • 0

I'm figuring out how to download fastq files from either SRA or ENA in a quick manner. I have previously done it on a HPC computer using prefetch/fastq-dump. However, when I tried this on my personal laptop, the process took a very long time.

I have a vague idea that Aspera Connect can provide high-speed downloading of fastq file. However, I'm quite perplexed as to which application to be downloaded (whether it is Aspera Connect or Aspera CLI or both?) and installed and the specific process of downloading fastq files with aspera.

aspera sratoolkit fastq • 481 views
ADD COMMENT
1
Entering edit mode

Download aspera client from: https://www.ibm.com/aspera/connect/

(from @ATPoint tutorial found elsewhere on biostars, not linking directly since it is deprecated in parts):

For Linux, it is a tarball (use tar zxvf to unpack) with an installer batch script and for Mac, a standard disk image.

After installation, there now will be these executables/files in their default locations:

Linux:

$HOME/.aspera/connect/bin/ascp --- the executable

$HOME/.aspera/connect/etc/asperaweb_id_dsa.openssh --- openssh file

Mac:

$HOME/Applications/Aspera\ Connect.app/Contents/Resources/ascp --- the executable

$HOME/Applications/Aspera\ Connect.app/Contents/Resources/asperaweb_id_dsa.openssh --- openssh file

In any case, make sure you add the folder with the ascp executable to your $PATH.

Then use sra-explorer to get the aspera command lines.

Edit: Going to move this to a comment since the installation on linux does not seem to be functional. As noted by ATpoint the SSH key file is missing from installer.

ADD REPLY
0
Entering edit mode

Just realized that with the current version of Aspera Connect (>4.2) they removed that openssh key from the tarball one can download -- and without that the command does not work anymore, older versions are behind a login wall, and I do not know how to work around that for now. If anyone knows please share.

Or just use wget links...

ADD REPLY
0
Entering edit mode

IBM appears to have made this more difficult than necessary. Now the openssh keys may be provided by individual clients. So unless you download the client from a place that licenses aspera this may no longer work.

ADD REPLY
0
Entering edit mode

Found a way -- writing a tutorial, hang on.

ADD REPLY
0
Entering edit mode

Not a direct reply to your question: maybe you can give gget a try? I never tried it, and I'd be surprised if it was a drop in replacement for aspera. For myself I had not the patience to setup aspera on my side.

ADD REPLY
0
Entering edit mode

I don't think gget can download data from SRA.

ADD REPLY
1
Entering edit mode
6 weeks ago

What I usually do is to go via sra-explorer.info , that site is excellent to search for datasets and to then get several options on how to download them ... ftp, wget, ...

ADD COMMENT
0
Entering edit mode
ADD COMMENT

Login before adding your answer.

Traffic: 1072 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6