Question: How do you use the Aspera ascp command?
2
gravatar for cyril-cros
4.5 years ago by
cyril-cros890
France
cyril-cros890 wrote:

Hi, I would like to download a large dataset from GigaScience. The FTP speed is a steady 70ko/s, and I have 20Gb of data to get.

I need to use Aspera, which I have installed and tested with the company demo. The Connect plugin (3.6) fails to work on Windows/Linux with Chrome/Firefox/Edge. I would rather try the ascp command line utility.

Question: how does it work? I have spent three hours reading various tutorials and tinkering. I have located the ascp program, its log files, and some other users command on the Internet.

All I get is "ascp: failed to authenticate, exiting.", and no log files. If you are familiar with ascp, could you please post the command line you are using? Did you generate some sort of SSH key for authentification?

PS: Please pretty please don't refer me to another post unless its solution really works 100% for you. There are lots of links to Aspera, a section in the GigaDB help page, but they are all terribly imprecise. GigaDB refers you to Aspera, which has rather esoteric usage examples.

PS: some sample commands from online posts and their ouput:

~/.aspera/connect/bin/ascp -v -P 3867 -Q  gigadb@aspera.gigadb.org:/pub/10.5524/200001_201000/200001/rice3k/0001_1000/IRIS_313-10002/IRIS_313-10002_120220_I297_FCC0DJBACXX_L4_RICwdsRSYHSD24-3-IPAAPEK-84_2.fq.gz .
# Failure to authenticate

ascp -i aspera/connect/etc/asperaweb_id_dsa.openssh -Tr -Q -l 10M -L- fasp-g1k@fasp.1000genomes.ebi.ac.uk:vol1/ftp/release/20100804/ALL.2of4intersection.20100804.genotypes.vcf.gz ./ 
# Failure Event: -16 - Unable to open private key file ...
ERR [ascp] SSH authentication failed

 

aspera gigadb gigascience • 8.4k views
ADD COMMENTlink modified 4.5 years ago • written 4.5 years ago by cyril-cros890

well does this file exist?

aspera/connect/etc/asperaweb_id_dsa.openssh

or is it

.aspera/connect/etc/asperaweb_id_dsa.openssh
ADD REPLYlink modified 4 months ago by RamRS26k • written 4.5 years ago by Jeremy Leipzig19k

The filename should in fact be asperawebid_dsa.openssh, thanks for your input. I still get the same message:

LOG Aspera Connect version 3.6.1.110647
LOG Alternate log directory: "-"

LOG Configuration: using v2 configuration file "/home/cyril/.aspera/connect/etc/aspera.conf", user -
LOG Initializing FASP version 3.5.6.110366, license max rate=(unlimited), account no.=1, license no.=1 product=6
LOG Configured symlink actions: create=1, follow=1, follow_wide=0, skip=0
LOG [asssh] remote host-key fingerprint 3eb78b35f5a01fdf131b20255d689196dcae1efa
LOG [libssh2] 0.365553 Failure Event: -16 - Unable to open private key file
LOG [libssh2] 0.365569 Failure Event: -16 - Unable to extract public key from private key file
ERR [ascp] SSH authentication failed
ERR failed to authenticate
ADD REPLYlink modified 4 months ago by RamRS26k • written 4.5 years ago by cyril-cros890

and what does it say when you replace your key file path with garbage? the same error?

ascp -i somenonexistentpath -Tr -Q -l 10M -L- fasp-g1k@fasp.1000genomes.ebi.ac.uk:vol1/ftp/release/20100804/ALL.2of4intersection.20100804.genotypes.vcf.gz ./
ADD REPLYlink modified 4 months ago by RamRS26k • written 4.5 years ago by Jeremy Leipzig19k

Still the same error, weirdly enough... But I don't really care about encryption when downlaoding public data, I mught as well just look at the GigaDB command line example. It fails the same way it seems, but there are no verbose logs...

ADD REPLYlink modified 4.5 years ago • written 4.5 years ago by cyril-cros890

I still think you're missing the dot. Make sure that key file exists where you say it does.

ADD REPLYlink written 4.5 years ago by Jeremy Leipzig19k

any updates?

ADD REPLYlink written 4.5 years ago by Jeremy Leipzig19k

I contacted the GigaDB database managers. I will get some feedback tomorrow, but the GigaDB website was down yesterday.

Running

~/.aspera/connect/bin/ascp -v -QT -l20M -i ~/.aspera/connect/etc/asperaweb_id_dsa.openssh anonftp@ftp.ncbi.nlm.nih.gov:/1GB .

worked. However, I failed to use a similar version for the GigaDB website.

ADD REPLYlink modified 4 months ago by RamRS26k • written 4.5 years ago by cyril-cros890

Hi Cyril,

Did you ever get anywhere with this? Am experiencing the same issue.

ADD REPLYlink written 4.4 years ago by Andrew Conkie0

Hi Andrew,

GigaDB encountered some hardware issues while I was trying to make Aspera work. I ended up downloading a large file (~30Gb) by FTP, over 4 days. It was in fact corrupted. GigaDB does not include checksums for its files....

I ended up contacting the corresponding authors, who were immensely useful. They gave me a link to their own files, and I had the 'real' data file (46Gb) within 3 hours using their in-house FTP server.

My advice is to spare yourself these issues and talk directly to the authors.

ADD REPLYlink written 4.4 years ago by cyril-cros890

Thanks Cyril, may end up going that way.

Appreciate your help.

ADD REPLYlink written 4.4 years ago by Andrew Conkie0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 861 users visited in the last hour