How do you use the Aspera ascp command?
0
2
Entering edit mode
9.1 years ago
cyril-cros ▴ 950

Hi, I would like to download a large dataset from GigaScience. The FTP speed is a steady 70ko/s, and I have 20Gb of data to get.

I need to use Aspera, which I have installed and tested with the company demo. The Connect plugin (3.6) fails to work on Windows/Linux with Chrome/Firefox/Edge. I would rather try the ascp command line utility.

Question: how does it work? I have spent three hours reading various tutorials and tinkering. I have located the ascp program, its log files, and some other users command on the Internet.

All I get is ascp: failed to authenticate, exiting., and no log files. If you are familiar with ascp, could you please post the command line you are using? Did you generate some sort of SSH key for authentification?

PS: Please pretty please don't refer me to another post unless its solution really works 100% for you. There are lots of links to Aspera, a section in the GigaDB help page, but they are all terribly imprecise. GigaDB refers you to Aspera, which has rather esoteric usage examples.

PS: some sample commands from online posts and their ouput:

~/.aspera/connect/bin/ascp -v -P 3867 -Q gigadb@aspera.gigadb.org:/pub/10.5524/200001_201000/200001/rice3k/0001_1000/IRIS_313-10002/IRIS_313-10002_120220_I297_FCC0DJBACXX_L4_RICwdsRSYHSD24-3-IPAAPEK-84_2.fq.gz .
# Failure to authenticate
ascp -i aspera/connect/etc/asperaweb_id_dsa.openssh -Tr -Q -l 10M -L- fasp-g1k@fasp.1000genomes.ebi.ac.uk:vol1/ftp/release/20100804/ALL.2of4intersection.20100804.genotypes.vcf.gz ./
# Failure Event: -16 - Unable to open private key file ...
ERR [ascp] SSH authentication failed
aspera gigascience gigaDb • 19k views
ADD COMMENT
0
Entering edit mode

well does this file exist?

aspera/connect/etc/asperaweb_id_dsa.openssh

or is it

.aspera/connect/etc/asperaweb_id_dsa.openssh
ADD REPLY
0
Entering edit mode

The filename should in fact be asperawebid_dsa.openssh, thanks for your input. I still get the same message:

LOG Aspera Connect version 3.6.1.110647
LOG Alternate log directory: "-"

LOG Configuration: using v2 configuration file "/home/cyril/.aspera/connect/etc/aspera.conf", user -
LOG Initializing FASP version 3.5.6.110366, license max rate=(unlimited), account no.=1, license no.=1 product=6
LOG Configured symlink actions: create=1, follow=1, follow_wide=0, skip=0
LOG [asssh] remote host-key fingerprint 3eb78b35f5a01fdf131b20255d689196dcae1efa
LOG [libssh2] 0.365553 Failure Event: -16 - Unable to open private key file
LOG [libssh2] 0.365569 Failure Event: -16 - Unable to extract public key from private key file
ERR [ascp] SSH authentication failed
ERR failed to authenticate
ADD REPLY
0
Entering edit mode

and what does it say when you replace your key file path with garbage? the same error?

ascp -i somenonexistentpath -Tr -Q -l 10M -L- fasp-g1k@fasp.1000genomes.ebi.ac.uk:vol1/ftp/release/20100804/ALL.2of4intersection.20100804.genotypes.vcf.gz ./
ADD REPLY
0
Entering edit mode

Still the same error, weirdly enough... But I don't really care about encryption when downlaoding public data, I mught as well just look at the GigaDB command line example. It fails the same way it seems, but there are no verbose logs...

ADD REPLY
0
Entering edit mode

I still think you're missing the dot. Make sure that key file exists where you say it does.

ADD REPLY
0
Entering edit mode

any updates?

ADD REPLY
0
Entering edit mode

I contacted the GigaDB database managers. I will get some feedback tomorrow, but the GigaDB website was down yesterday.

Running

~/.aspera/connect/bin/ascp -v -QT -l20M -i ~/.aspera/connect/etc/asperaweb_id_dsa.openssh anonftp@ftp.ncbi.nlm.nih.gov:/1GB .

worked. However, I failed to use a similar version for the GigaDB website.

ADD REPLY
0
Entering edit mode

Hi Cyril,

Did you ever get anywhere with this? Am experiencing the same issue.

ADD REPLY
0
Entering edit mode

Hi Andrew,

GigaDB encountered some hardware issues while I was trying to make Aspera work. I ended up downloading a large file (~30Gb) by FTP, over 4 days. It was in fact corrupted. GigaDB does not include checksums for its files....

I ended up contacting the corresponding authors, who were immensely useful. They gave me a link to their own files, and I had the 'real' data file (46Gb) within 3 hours using their in-house FTP server.

My advice is to spare yourself these issues and talk directly to the authors.

ADD REPLY
0
Entering edit mode

Thanks Cyril, may end up going that way.

Appreciate your help.

ADD REPLY

Login before adding your answer.

Traffic: 1982 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6